Method for classifying a multitude of images recorded by a camera observing a processing area and laser material processing head using the same

ABSTRACT

The present invention relates to a method for classifying a multitude of images recorded by a camera observing a processing area of a workpiece processed by a processing beam, comprising the steps of: recording a first pixel image and a multitude of subsequent pixel images by the camera during a processing operation; detecting mismatches of a position and orientation of a keyhole generated by the processing beam in the workpiece within an image plane of the subsequent pixel images in comparison to the first pixel image; compensating the mismatches of the position and orientation of the respective keyholes in the subsequent pixel images with regard to the first pixel image, to produce a set of pixel images having each a normalized keyhole position and orientation; classifying the set of normalized pixel images into at least two classes by means of a classifier.

The present invention relates to a method for classifying a multitude ofimages recorded by a camera observing a processing area of a workpieceprocessed by a processing beam and to a laser material processing headusing the same.

In industrial nations with high labor and living costs, it is importantto increase automation in manufacturing in order to retain a competitiveedge. Furthermore, there is an ongoing trend from mass production towardincreased flexibility in product variation while maintaining high outputvolumes. Cognitive capabilities for production machines may improve withflexibility and automation to contribute to a Cognitive Factory of thefuture. Artificial software agents, systems with cognitive capabilities,subsequently just agents, may help to tailor products individually andto deliver them on a large scale. Furthermore, due to a computer'ssuperior skills in data analysis, agents may be able to manage complexproduction tasks that are challenging even for human experts. A possibletest scenario for these agents could be an upcoming production methodthat is complex to handle and needs improvement in terms of flexibility.Therefore laser material processing may be a good choice forinvestigating the cognitive capabilities of artificial agents inperforming production tasks.

Treating materials with laser beams has several advantages. The laser isone of the highest density energy sources available to industry. Onceconfigured, a laser processing system works with extraordinaryprecision, enabling high cut quality in laser cutting or deep and thinweld seams in laser beam welding. Therefore laser material processing isalready frequently applied in a great variety of production processes,mostly out of the public view. Automotive manufacturers apply laserprocessing in many steps of car body production, but laser processing isalso used for consumer and medical products such as household devices orcoronary stents. However, users must expend a great deal of cost andeffort on trials before a laser processing system can run. For everydesired change in the processing task, the user may have to repeat theconfiguration procedure. Even if all process parameters remainuntouched, slight differences in workpiece, workload, alignment, roomtemperature, or optical properties may result in a loss of quality and,in the worst case, a suspension of the assembly line. Laser cutting andlaser welding may thus benefit from the cognitive capabilities ofartificial agents. If these agents can learn how to weld or cut, itwould not only reduce the system configuration effort, but also increaseits flexibility. Moreover, if an agent could improve itself over time,it could gain the capability to develop its everyday tasks, increaseoutput, and assure quality. Many manufacturers wish to have a promptcutting or welding technique, a system that does not need to bereconfigured when it takes over a new production task. This kind ofsystem would significantly increase welding and cutting efficiency andassure quality. Quality assurance is especially important whenprocessing parts are associated with safety, for instance within cars orairplanes.

Another advantage of process control, besides increased quality andflexibility, would be to save environmental resources. For instancelaser cutters use higher laser power than necessary as a safety marginto maintain a minimum kerf width and to prevent a loss of cut.Artificial software agents might learn to apply just enough energy foroptimal cutting efficiency, thus saving energy with every cut. Forexample, five 8 kW fiber lasers with a wall plug efficiency of 30%integrated in an assembly line in Germany operating 253 days a year, 18hours a day, create operational electricity costs of over US$50,000annually. With 32 g CO2 emission per kWh, this adds up to approximately20 metric tons annually. In addition, fiber lasers apparently have abetter wall plug efficiency compared to other common industrial lasersources, which are sometimes less efficient by a factor of 15. Takingonly a factor of six, this would add up to operational electricity costsof US$300,000 and over 100 metric tons of annual CO2 emission. If anartificial agent manages to save just 10% in laser power, this may saveup to US$30,000 and approximately 2 to 10 or more metric tons of annualCO2 emission within just one sample assembly line. It however remainsunclear whether it is possible to define a cognitive architecture thatcan create artificial agents that can learn tasks from laser cutting orwelding and can then reliably monitor and control in real-time, improveprocessing, and save resources.

State-of-the-Art Scientific and Industrial Approaches

In general, laser material processes are established and configuredthrough a series of trials. Reference tests are carried out until ahuman expert has found a possible parameter set. In welding, the weld isanalyzed with microscopic pictures of a cross-section of the seam.Finally, once the user finds successful parameter sets, the parametersremain untouched and any process disturbance is excluded if possible.Because this process involves high effort and cost, manufacturers oftendeclare the parameter sets to be classified. However, even if everyattempt is made to keep all process parameters constant, slight changesand nonlinear behavior can result in poor cutting quality or weldingdefects. For quality assurance, many industrial users need to implementmonitoring systems to observe their laser processes.

Monitoring in Laser Material Processing

There are two general monitoring standards in laser cutting: maintaininga minimum kerf width and a certain cutting edge quality. Problems incutting edge quality include, for instance, dross, roughness, orparallelism of edges. The overall quality or variation in edge roughnessis determined by many parameters such as room environment, gas andnozzle parameters, focus position, laser power, feed rate, angle orradius parameters, laser beam conditions and alignment, the metal alloy,surface coatings, among many others. A welding seam may have undesiredsurface irregularities, including breaks, holes, material ejections, theformation of spatters, cracks, pores, seam width variation, and manymore. Sophisticated monitoring systems have thus been introduced forindustrial laser welding to detect the problems listed; there are threetypes: pre-, in-, and post-process monitoring. A number of publicationshave emphasized that in-process or online process monitoring may detectwelding defects. On top of these, there may also be welding errors, suchas an undesired degree of welding depth or insufficient connection,which often cannot be observed without destroying the welding seam. Thelatter may expand and lead to a complete lack of fusion. A lack offusion involves a gap between the partners that should have been joined.The gap is often visible neither from the top nor from the bottom of thewelded workpiece and is therefore called a false friend.

A frequently used sensor for monitoring laser cutting as well as weldingis a camera aligned coaxially to the laser beam axis. Such a camera cancapture images of the heat affected zone and the treated workpiece. Itmay also be suitable for closed-loop process control. Related researchindicates that a coaxially observing camera can allow monitoring of theappearance of dross and the existence of an insufficient cut or minimumkerf width. An illumination unit may significantly improve monitoringwith cameras because the workpiece surface and more details of the heataffected zone are visible. The coaxially integrated camera is a veryvaluable sensor for monitoring cutting and welding processes andproviding temporally and spatially resolved process information. Asmentioned above, detecting false friends is a difficult monitoring taskbecause the weld seam may appear defect-free from the outside at thelocation of a lack of fusion. The defect may, for example, be monitoredwith thermal imaging. When observing a thermal image of an integratedcamera, the heat dissipation seems to be longer towards the cooling weldseam if a false friend is present. A temperature ratio between twopositions of the cooling weld seam, one closer to the focal spot thanthe other, may detect the presence of a lack of fusion in some cases. Onthe contrary, this method seems to require costly equipment and theconfiguration of fixed measurement positions. This method has to behowever manually adapted to the individual processes.

In an analysis of the optical spectrum of process emission in laserwelding or cutting, differences in distribution and intensity coincidewith process changes. The same seems to be true for air-borne orsolid-borne acoustic emissions. Optical as well as acoustical emissionsseem to deliver similar process information. Wavelength filteredphotodiodes often capture information on specific spectral processemissions. Many users thus apply three photodiodes, respectivelysensitive to laser back reflection, temperature, and plasma plumeemissions.

However, it is hardly possible to cover all of the effects in lasermaterial processing with just one kind of sensor technology. Thus,combining several sensor signals for improved monitoring of lasermaterial processing has several advantages.

Closed-Loop Control in Laser Material Processing

The vast majority of industrial laser material processing applicationsare manually configured and supervised. It is economically worthwhile todecrease human labor costs and system downtime for laser processingsystems. As a result, it has been a long-term research goal to achieveclosed-loop control of at least one influential process parameter. Someparameters of laser material processing have a short response time and agreat influence on the process outcome. Therefore, these parameters havebeen subdivided into two groups: distance control and laser powercontrol.

As stated above, receiving a failure-free monitoring signal in lasermaterial processes is challenging in both laser welding and lasercutting. Nevertheless, many theories have been developed and somespecialized systems are now used in industrial environments, such ascapacitive distance control in laser cutting. The capacitive distancecontrol works reliably in many industrial applications to maintain aconstant distance between the workpiece and the processing head.

Some attempts have been made to attain closed-loop laser power control;for example, a laser power is controlled by a threshold function for aprocess emission photodiode. This method made it possible to find afixed relation between weld speed and laser power. Photodiode signalsmay vary significantly with slight process parameter changes. Thereforecontrol methods with static functions of photodiode intensity to laserpower suffer from process disturbances. A photodiode mounted at thebottom of the welded workpiece detects different intensities dependingon the degree of workpiece penetration. These root side light emissionscontrol the laser power within a closed loop. For many industrialapplications, this method is not suitable because the root side of theworkpiece is not accessible. Furthermore, this method only works forfull penetration welds when the laser beam exits the work-piece at theroot side. Closed-loop control of laser power and focal position hasalso been studied. In this case, a fixed threshold for keyhole openingat a fixed position that controls the laser power and the focal positionis altered with changes in chromatic aberration. The keyhole openingseems to be a significant camera picture feature suitable for fullpenetration welding. However, many welding processes do not have avisible keyhole within the camera image. Often the keyhole is onlyvisible in full penetration welding with high laser power, resulting insignificant heat conduction within the workpiece and excessivepenetration with weld seam root convexity.

Besides using a processing head that works relatively close to theworkpiece, it is possible to use so-called scanners with beam guidingmirrors for remote welding applications. Monitoring systems for remotewelding is a promising topic for future research. An approach for laserpower control within remote welding has been demonstrated withsophisticated experimental results. An algorithm finds a keyhole withina camera equipped with a Cellular Neural Network environment. A controlloop increases the laser power until a keyhole is detected within thecamera picture and maintains it at a constant size. However, as statedabove, a keyhole is only visible within the camera picture when there isvery high laser power resulting in significant heat influence on theworkpiece. Furthermore, only full penetration welding is possible withthis technique, but is not desired in every case.

Closed-loop control seems to be a highly complex task for laser materialprocessing. Most monitoring signals merely give relative feedback ratherthan absolute values. Small changes in the distance between theworkpiece and the processing head may result in different absolutevalues for monitoring signals, but with the same process result. Theproposed approaches for closed-loop control seem to be suitable only fordefined process modes such as full penetration welding with a high levelof laser power or fixed thresholds. A possible cure for a closed-loopcontrol system would add increased adaptability, as will be discussed inthe next section.

Adaptive Control and Monitoring Approaches in Laser Material Processing

With the many quality control and closed-loop control systems that havebeen explored in the literature, there must be some reason why only afew are applicable for industrial use. One reason may be that thesesystems only work for individual applications but are not suitable tocover a great variety of different processes. An enhanced adaptabilitymay be a solution to this problem. If a system can learn how to adapt toa certain number of distinct applications, this may already be morevaluable for manufacturing purposes. Moreover, it appears that an idealsensor that always gives accurate and absolute information about theprocessing state has not been found for laser material processing. Anevaluation of multiple sensor data input may help to improve themonitoring results and better to grasp the system's state. In this way,many sensor data inputs with individual weaknesses may be combined tobecome more reliable, in the same way that humans rely on several sensesto make judgments. Thus cognitive capabilities may help to bridge theexisting gap and apply laser material processing to more manufacturingprocesses, increase quality performance, and decrease wastage ofresources.

Several sophisticated approaches using methods from machine learning orwith cognitive capabilities have already been discussed in theliterature. The general idea of an autonomous production cell for laserbeam welding has been investigated. Other approaches may be subdividedinto systems that combine one or more sensors intelligently to monitorthe process, and approaches that aim to control the process.

Recent techniques in machine learning and the control of laser beamwelding have been examined to create adaptive monitoring. ArtificialNeural Networks (ANN), Support Vector Machines (SVM), and the FuzzyK-Nearest Neighbor (KNN) classification have been investigated as theyapply to special applications for laser material processing.

In order to control the welding speed, a method of defining thresholdswith fuzzy logic rules has been provided. This is studied in combinationwith a fuzzy logic process control. Here, the process information isfirst analyzed statistically before it is used for closed-loop controlto cope with the fact that the information gained about the process isweak for closed-loop controlling purposes. Related work using an expertsystem can be found. ANN for laser material processing purposes havebeen investigated. An ANN is investigated to create a predictive processmodel of optical process emission, welding speed, laser power, and focalposition, which is then adapted to the process. This is a promisingapproach, but the necessity of first defining a process model createsadditional effort. One aim of the present invention is to evaluate whatmachine learning can accomplish without a process model.

Although there has been significant scientific interest in finding anadaptive system that can manage different tasks in laser materialprocessing, it seems as if this step still needs to be taken. Either thediscussed approaches do not include experimental data or they seem to besuitable only for specific applications.

In summary, laser material processing systems require a major effort ininstallation and reconfiguration. Typically, the systems are set up toexecute a specific task in the same way again and again. The current aimfor these systems is to keep all of the influential parameters constant,which is often not the case in real industrial applications. Materialsvary from piece to piece or from one workload to the next. The mountingmay not be the same all the time because of variations resulting fromeither human labor or imprecise robots. However, there is a great desirefor fault-free weld seams and stable cutting quality. This results notonly from a need to optimize manufacturing economically or to conserveenvironmental resources, but also because this is a major safety issue,especially for car or airplane bodies. This means that quality controlis essential, along with, ideally, closed-loop process control systemsthat are able to work reliably in the demanding environment of materialprocessing with high-powered laser beams. It seems as if these goalshave not been met by the current state of research, as is describedabove.

SUMMARY OF THE INVENTION

From the current state of the art there is a gap to be bridged in lasermaterial processing research in creating monitoring systems that areable to detect some kinds of cutting errors or welding defects. One stepforward would be to realize adaptive monitoring that is capable oflearning the reliable detection of a lack of fusion based on severalsensor signals.

Since laser material processing research is part of manufacturing, theresulting system should qualify and be of use for real industrialapplications. This leads to certain premises: the system has to berobust enough to cope with a high degree of adaptivity for differentlaser material processes; it has to execute its capabilities inreal-time for the processing task; it has to be user-friendly; and itssensors and components must be affordable for the purpose. The systemshould be autonomous, yet transparent to the human expert. Therefore,the design of the cognitive capabilities should enable demonstration ofthe system's actions and decisions to the human expert in order tosecure the best possible quality control.

The present invention should demonstrate reliable detection of a lack offusion. A successful closed-loop control method for the future shouldnot only be able to adapt the laser power to speed alterations, itshould also be applicable to at least two different processing tasks. Inother words, the agent should adjust the laser power to speed changes inorder to maintain a similar energy per unit length level within a set ofexperiments for similar welding or cutting results. In addition, inorder to investigate the cognitive capabilities of the system, it shouldbe able to learn from human experts, as well as show reasonable behaviorand continued learning from feedback in an unsupervised mode withinexperiments.

Thus, it is a main object of the present invention to take advantage ofcognitive capabilities in order to increase a production system inflexibility, quality, and efficiency. This can be further separated infour objects:

It is a first object of the present invention to provide a system beingable to gain knowledge by learning from a human expert how to abstractrelevant information within production tasks and how to weld or cut,wherein the system should show reasonable behavior in unknown situationsand should be able to learn unsupervised.

It is a second object of the present invention to provide a systemmaintaining quality with reliable detection of hard-to-detect defectssuch as false friends or a lack of fusion within experiments.

It is a third object of the present invention to provide a systemincreasing the efficiency by closed-loop control of laser power adaptingto changes in processing speed and maintaining penetration depth.

It is a fourth object of the present invention to provide a systemhaving flexibility for individually different processing tasks byadapting to different materials or process tasks.

These objects are solved by a method according to claim 1 and by thelaser material processing system according to claim 15. Furtheradvantages, refinements and embodiments of the invention are describedin the respective sub-claims.

The present invention seeks to examine ways of realizing cognitivecapabilities and improving workstations in manufacturing using lasermaterial processing systems. Cognitive capabilities could involveabstracting relevant information, learning how to laser weld or lasercut, and using the knowledge gained to deal with new but similar taskssuccessfully.

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1A illustrates a laser cutting process of melt and blow and acorresponding laser processing head of the present invention;

FIG. 1B shows a deep penetration laser beam welding process and acorresponding laser processing head of the present invention;

FIG. 1C shows a cognitive perception-action loop for production machineswith sensors and actuators according to the present invention;

FIG. 1D shows categories of linear and nonlinear dimensionalityreduction techniques;

FIG. 1E shows a mapping of two-dimensional test data to athree-dimensional space with an optimal linear separator;

FIG. 2A shows an architecture according to the present invention andcomponent groups to design agents for process monitoring or closed-loopcontrol in production systems using a black-box model with sensors andactuators;

FIG. 2B shows a cognitive architecture according to the presentinvention providing modules to design monitoring or control agents forlaser welding or cutting systems using a black-box model with sensorsand actuators;

FIG. 2C shows an in-process image taken with a coaxial camera (left) anda laser welding process with acoustic sensors (right);

FIG. 2D shows an embodiment according to the present invention withsensors and actuators for laser material processing;

FIG. 2E illustrates a process of the present invention for reducingin-process pictures to camera features: upper row, in-process picturesfor low, medium, and high laser power—middle row, PCA and LDA featureswith a mapping having bright areas with positive or negative variancevalues and black areas for variance values around zero—lower row,feature maps of middle row being illustrated schematically;

FIG. 2F shows monitoring signal values for a welding agent according tothe present invention detecting laser power gradient and soiledworkpieces using Artificial Neural Networks; monitoring signal valuesaround 0.5 indicate optimal laser power, smaller than that indicate toolow and greater values too high laser power;

FIG. 3 shows an embodiment of real-time closed-loop control architectureof the present invention.

FIG. 4 shows a mapping of laser cutting features according to thepresent invention from a reference workpiece onto workpieces with achange in thicknesses from 1.2 mm to 0.7 mm stainless steel; coloredzone indicates trained classes and the new areas for additional featurelearning;

FIG. 5 shows a training workpiece for a monitoring agent according tothe present invention with feature values: zinc-coated steel, gapvariation from 0 mm to 1.0 mm to 0 mm for a lack of fusion detectionwith temperature, PCA (camera feature 1 and 2), LDA (camera feature 3),and Isomap (camera feature 4 and 5) feature values vs. processingtime—regions I and III indicate the training areas for a lack of fusion(III) and existing connection (I)—a region II indicates the window of anoccasional lack of fusions—upper picture row shows coaxially takenin-process camera pictures, with connection left and right and withoutconnection in the middle—middle picture row displays the eigenvectorswith a map with bright areas for positive and negative values and blackareas for values around zero—lower row, feature maps of middle row beingillustrated schematically;

FIG. 6 shows Z001 zinc-coated steel workpiece agent according to thepresent invention monitored with SVM, Fuzzy KNN, and ANN classificationresults: two inserted gaps of 1.0 mm at the middle on the left side(left area III) and 0.6 mm at the middle of the right side (right areaIII) of the workpiece—workpiece has existing connection at the area Iand a lack of fusion at areas III from 1.2 s to 4.2 s and 6.2 s to 8.6 sof processing time—the area II shows a lack of fusion at some positions;

FIG. 7 shows a Z002 zinc-coated steel workpiece agent according to thepresent invention monitored with SVM, Fuzzy KNN, and ANN classificationresults: zinc-coated steel, two inserted gaps of 0.6 mm at the far leftside (left area III) and 1.0 mm at the far right side (right area III)of the workpiece—workpiece has existing connection at the area I and alack of fusion at area III from 0 s to 3.2 s and 6.6 s to 10.0 s ofprocessing time—the area II shows a lack of fusion at some positions;

FIG. 8 shows a training workpiece for a cutting control agent accordingto the present invention: stainless steel, 1.0 mm material thickness,laser power ramp from 1,500 W to 50 W, robot velocity at 3 m/min vs.processing time—upper picture row shows coaxially taken in-processcamera pictures—middle picture row displays the eigenvectors with a mapwith bright areas for positive or negative values, and black areas forvalues around zero—lower row, feature maps of middle row beingillustrated schematically;

FIG. 9 shows CA001-CA006 workpieces being processed by control of acutting agent according to the present invention: stainless steel,closed-loop, 1.0 mm material thickness, laser power controlled atdifferent velocities from 1.8 m/min(CA001) to 7.8 m/min(CA006) vs.processing time-displayed averaged laser power values;

FIG. 10 shows kerf scans of CA001-CA006 workpieces being processed bycontrol of the cutting agent according to the present invention andCC002-CC006 workpieces being processed with constant laser power:stainless steel, 1.0 mm material thickness, laser power controlled atdifferent velocities from 1.8 m/min (CA001) to 7.8 m/min (CA006) vs.CC002-CC006 with 750 W laser power at corresponding velocities;

FIG. 11 shows a material variation of CA007, CA008, and CA002 workpiecescontrolled by a cutting agent according to the present invention trainedwith stainless steel: CA007 mild steel, 1.0 mm material thickness—CA007zinc-coated steel, 1.0 mm material thickness—CA002 stainless steel, 1.0mm material thickness—laser power controlled with features of stainlesssteel at 3 m/min velocity vs. processing time;

FIG. 12 shows a J001 workpiece with layered one, two, and three sheetscontrolled by cutting agent according to the present invention:stainless steel workpieces with stepwise 1, 2, or 3 sheet(s) with 0.6 mmmaterial thickness—laser power controlled at 3 m/min velocity vs.processing time;

FIG. 13 shows a training workpiece for a welding control agent accordingto the present invention: two overlapping stainless steel sheets with0.6 mm material thickness at the top sheet and 1.2 mm at the bottomsheet—cross-sections every 50 mm—laser power ramp from 1,000 W to 50 W,robot velocity at 1.8 m/min vs. processing time—upper picture row showscoaxially taken in-process camera pictures—middle picture row displaysthe eigenvectors with a map with bright areas for positive or negativevalues and black areas for values around zero—lower row, feature maps ofmiddle row being illustrated schematically;

FIG. 14 shows a second stainless steel training workpiece for weldingwith laser power ramp from 1,500 W to 50 W and a velocity of 1.2 m/minvs. processing time;

FIG. 15 shows WA001-WA007, WA010 workpieces controlled by a weldingagent according to the present invention at different velocities: twooverlapping stainless steel sheets with 0.6 mm material thickness at thetop sheet and 1.2 mm at the bottom sheet—WA010 two overlapping stainlesssteel sheets with 0.6 mm material thickness at the top sheet and 0.6 mmat the bottom sheet at velocity of 1.2 m/min—laser power controlled vs.processing time;

FIG. 16 shows scans and cross-sections of WA001-WA004 workpiecescontrolled by a welding agent according to the present invention atdifferent velocities: two overlapping stainless steel sheets with 0.6 mmmaterial thickness at the top sheet and 1.2 mm at the bottom sheet;

FIG. 17 shows scans and cross-sections of welding agent controlledworkpieces WA005-WA007, WA010 at different velocities: two overlappingstainless steel sheets with 0.6 mm material thickness at the top sheetand 1.2 mm at the bottom sheet—WA010 two overlapping stainless steelsheets with 0.6 mm material thickness at the top sheet and 0.6 mm at thebottom sheet at velocity of 1.2 m/min;

FIG. 18 shows a picture of penetration depth controlled WA001-WA007workpieces at different velocities;

FIG. 19 shows an additional training workpiece WA008 for learning how toweld with 50% less material thicknesses in the bottom sheet: stainlesssteel with 0.6 mm at top sheet and 0.6 mm at bottom sheet—the areasmarked with an arrow indicate human expert feedback, indicated by (−)for too much laser power—processing speed is 1.2 m/min;

FIG. 20 shows an additional training workpiece WA009 for learning how toweld with 50% less material thicknesses in the bottom sheet: stainlesssteel with 0.6 mm at top sheet and 0.6 mm at bottom sheet—the areasmarked with an arrow indicate human expert feedback, (−) for too muchlaser power—processing speed is 1.2 m/min;

FIG. 21 shows an online nonlinear velocity variation from 4.2 m/min to1.8 m/min to 6.6 m/min welding agent laser power controlled: stainlesssteel with 0.6 mm at top sheet and 1.2 mm at bottom sheet;

FIG. 22 shows an overview of laser power vs. processing speed forcutting and welding according to the present invention;

FIG. 23 shows a cognitive laser welding system design according to thepresent invention with reinforcement learning agent;

FIG. 24 shows a training workpiece for a classifier according to thepresent invention, lap weld two stainless steel sheets with a 0.6 mmthick top sheet and 1.0 mm thick bottom sheet at 1.2 m/min—classes A andB represent the initial human expert provided knowledge about good andpoor welds—upper picture row shows coaxially taken in-process camerapictures—middle picture row displays the eigenvectors with a map withbright areas for positive or negative values and black areas for valuesaround zero—lower row, feature maps of middle row being illustratedschematically;

FIG. 25 shows a workpiece RIL004a: a reinforcement learning agentaccording to the present invention learns how to lap weld two stainlesssteel sheets with a 0.6 mm thick sheet on top and 1.0 mm thick sheetunderneath at 1.2 m/min;

FIG. 26 shows a workpiece RIL004b: a reinforcement learning agentaccording to the present invention learns how to lap weld two stainlesssteel sheets with a 0.6 mm thick sheet on top and 1.0 mm thick sheetunderneath at 1.2 m/min, continuing with policy parameters learned fromworkpiece RIL004a;

FIG. 27 shows a workpiece RIL005: a reinforcement learning agentaccording to the present invention learns how to lap weld two stainlesssteel sheets with a 0.6 mm thick sheet on top and 1.0 mm thick sheetunderneath at welding speed of 1.8 m/min;

FIG. 28 shows a workpiece RIL006: reinforcement learning agent accordingto the present invention learns how to lap weld two stainless steelsheets with a 0.6 mm thick sheet on top and 1.0 mm thick sheetunderneath at welding speed of 0.6 m/min; RL agent activated with delay;

DESCRIPTION OF THE EMBODIMENTS OF THE PRESENT INVENTION Aspects of LaserProcessing Systems and Technical Cognition

In the following, a brief overview of the theories underlying thepresent invention is given. This includes laser designs, modeling oflaser welding as well as cutting processes, and techniques for reducingsensor data with dimensionality reduction, such as Principal ComponentAnalysis, Linear Discriminant Analysis, and Isometric Feature Mapping.It also includes an introduction of classification and supervised aswell as unsupervised learning methods such as Fuzzy K-Nearest Neighbor,Artificial Neural Networks, Support Vector Machines, and reinforcementlearning. For the number format, the thousand separator is a comma “,”and the decimal separator is a point “.”; thus, one-thousand isrepresented by the number 1,000.00.

A single italic letter a indicates a variable on

a bold letter a=(a₁, . . . , a_(n))^(T) indicates a vector with ndimensions on

, and a capital italic letter A indicates a matrix. The termab=a·b=Σ_(i=1) ^(n)a_(i)b_(i) is defined as the dot product. TheEuclidean norm on

^(n) is defined as ∥a∥=√{square root over (a₁ ²+ . . . +a_(n) ²)}.

A data set can be represented by a t×n matrix X with the data elementsx_(i) as a vector or as value x_(ij). In all data analysis sections, weassume that the data is centered, which means that the mean of eachcoordinate over the entire data set is zero,

${\frac{1}{t}{\sum\limits_{i = 1}^{t}x_{ij}}} = 0$for every j=1 . . . n. The empirical covariance matrix cov(X) of a zeromean data set X is defined by

${{cov}(X)} = {\frac{1}{n}X^{T}{X.}}$The trace of a square matrix A with size n is defined as tr(A)=Σ_(i=1)^(n) a_(ii). The identity matrix of size n is defined as I_(n).

The expression p(ε) indicates the probability of an event ε. Thefunction

${p\left( {ɛ❘\xi} \right)} = \frac{p\left( {ɛ\bigcap\xi} \right)}{p(\xi)}$is the conditional probability of event ε, given the occurrence of someother event ξ. The expectation or the expected value becomesE(X)=Σ_(i)x_(i)p_(m)(x_(i)), if X is a discrete random variable withprobability measure p_(m).

Laser Material Processing

Laser material processing (LMP) refers to many well-establishedindustrial production techniques. Two important areas within these arelaser cutting and welding. The treatment of materials with laser beamshas several advantages over other production techniques in terms ofprecision, efficiency, and abrasion. Therefore LMP is applied withinmany areas of manufacturing, from tailored blanks in the automotiveindustry to small consumer products such as coffeemakers. However, LMPoften requires long configuration times and is highly sensitive tochanges in process parameters. In order to ensure quality, sophisticatedapproaches to observe cutting or welding processes are being implementedin industrial use. A common issue for LMP sensor systems is strongradiation including heat and spatter as well as nonlinear processbehavior. Therefore the processes in LMP are hard to observe anddifficult to control. One of the central components within LMP is ofcourse the laser itself.

High-Powered Industrial Laser Designs

A laser can be differentiated by the active medium: solid, liquid, orgas. Manufacturing companies often favor three types of lasers forindustrial processing: CO₂, Nd:YAG, and fiber lasers. CO₂ lasers operateat a wavelength of 10,600 nm.

We refer to Nd:YAG and fiber lasers as solid-state lasers. Solid-statelasers may create a pulsed output beam or a continuous wave (CW). Nd:YAGlasers operate at a wavelength of 1,064 nm. This investigationincorporates a fiber laser emitting a wavelength of 1,070 nm.

Laser Applications: Cutting and Welding in Manufacturing

Two of the common commercial applications of high-powered laser systemsare laser cutting and laser welding. Laser beams can be used in manydifferent ways to join or cut materials. The present invention focuseson common scenarios within industrial laser material processing: fusioncutting and deep penetration laser beam welding of metals with a fiberlaser in continuous wave mode and a processing head mounted on a roboticmoving device.

According to an embodiment of the present invention, for laser cutting,such as fusion cutting, a laser cutting head 100 (FIG. 1A) is provided,which directs a laser beam 102 to a workpiece 104 for cutting the same.The laser beam 102, which is focused by a lens system 106, heats thematerial at the desired spot at an energy density magnitude of severalMW/cm² at which it starts melting. A processing gas 108 flowing out froma nozzle 110 blows the melted material out. For this reason, fusioncutting is also referred to as melt and blow. Many parameters influencethe cutting results: the laser beam's focal spot size or position, aswell as polarization and wavelength; the processing speed v_(c), jetvelocity, nozzle position, and its shape and alignment, as well as thekind of gas; the composition material, its thickness, and itsreflectivity; and many other parameters. Manual trials are necessary toascertain the proper parameters before a parameter set for successfulprocessing can be derived. Even if in this case every attempt is made tokeep all parameters constant for future processing, the cut quality maybecome instable due to preconditions. Cutting errors such as dross,rough cutting edges, undesired kerf width variation, or a loss of cutmay occur when parameters such as the applied laser power or processinggas fail to meet the optimum level. To prevent this from happening,industrial users often keep a safety margin, using higher laser powerand gas pressures than necessary. Monitoring the laser cutting qualityhas the potential to increase output and conserve energy resources byreducing the safety margin.

According to the present invention, in order to monitor or control lasercutting, different sensor systems are applied. The laser cutting head100 applies an industrial process control system involving a capacitivemeasurement of the distance to be maintained between the workpiece 104and the laser cutting head 100. The sensor system is further adapted tocapture process emissions, and thus incorporates photodiodes, cameras,or acoustic sensors. Acoustic sensors are provided to detect gaspressure variation or eventually a loss of cut. Camera sensors have thepotential to detect the kerf width k_(w) and other cutting qualityparameters. However, because there are a great number of differentcutting processes, significant effort is often required before the usercan manage the desired cutting task and additional engagement is thennecessary to control its quality, if possible.

As a simple model for laser cutting, it is possible to find a heatcapacity equation based on the heat balance of the material removed bymelt and blow. Basically, the assumption is that the workpiece'sabsorbed energy per unit length is removed with the volume of meltedmaterial, assuming that further conducting into the workpiece isnegligible. The following equation describes the laser power to cuttingspeed ratio

$\frac{P_{L}}{v_{c}m_{z}}$as shown

$\begin{matrix}{\frac{P_{L}}{v_{c}m_{z}} \approx {\frac{k_{w}\rho}{c_{e}}{\left( {{C_{p}\Delta\; T} + L_{f} + {f_{v}L_{v}}} \right).}}} & \left( {{Formula}\mspace{14mu} 2.1} \right)\end{matrix}$

The parameter P_(L) describes the incident laser power, the variablev_(c) describes the cutting speed, and the material thickness is m_(z).Further parameters are the average kerf width k_(w), the materialdensity ρ, the coupling coefficient c_(e), the material's heat capacityC_(p), temperature rise to cause melting ΔT, latent heat of fusionL_(f), fraction of melt vaporized f_(v), and latent heat of vaporizationL_(v).

According to another embodiment of the present invention, in deeppenetration laser beam welding, or simply laser welding, a lasermaterial processing or laser welding head 200 (FIG. 1B) is provided,which directs a laser beam 202 to joint partners 204 a and 204 b forperforming a laser welding process. The laser beam 202 joins themultiple parts with concentrated but carefully dosed heat to achievemaximal connection within the joint partners 204 a, b. The heat affectedzone 206 is also referred to as an interaction zone and shows certainproperties. Often a robot moves the processing optics at a definedprocessing speed v_(w) in such a way that a desired weld seam 208 withthe best possible connection results after cooling. To this end, someprocesses incorporate a shielding gas 210. However, if the laser poweris too low, the welding process may suffer from a lack of penetration,or from so-called dropouts when it is too high. This may occur quicklywhen one of the process parameters varies. Most of the above mentionedinfluencing parameters for laser cutting also apply to laser welding.Additionally, compared to laser cutting, in welding there is increasedcomplexity stemming from highly influential process parameters such asgap tolerance or material composition. Furthermore, there is a multitudeof different joint geometries in welding. A lap weld is a joint of twooverlapping partners, while a butt joint describes two members alignedapproximately in one plane. Others are corner, edge, or t-joints, toname just a few. Of course each geometry has to be treated differentlyin terms of welding system setup and the interpretation of monitoringsignals.

In order to monitor deep penetration, also called keyhole laser welding,an operator normally measures the process emissions to draw conclusionsfrom them. When observing the heat affected zone from above duringprocessing, a melt pool 212 and a keyhole 214 as well as a plasma plume216 or metal vapor radiation may be visible. The melt pool 212 is thearea with melted material, within which is the keyhole 214, where thematerial is vaporized. The energy induction may create a radiatingionized gas of metal vapor or a plasma plume 216, depending on the laserbeam wavelength and material combination. This creates process emissionssuch that there are also temperature radiation and laser back reflectionthat can be captured with photodiodes. A coaxially integrated camera mayobtain the spatial resolution of the keyhole 214 and melt pool 212. Theaforementioned sensors are part of a group considered as in-processsensors. Another group are pre-process sensors, which often track thedesired weld spot position before the process takes place. Post-processsensors generally detect the weld seam geometry and surface. Monitoringsystems incorporating pre-, in-, and post-process sensors have increasedreliability to detect faults within the welding processes. However, evenexperienced welding experts sometimes cannot connect the welding resultswith the sensor signals, and some welding defects such as insufficientconnection may remain undetected.

As in the cutting part, it is possible to derive a form of lumpedheat-capacity model. The model describes the laser power and cuttingspeed ratio using

$\begin{matrix}{\frac{P_{L}}{v_{w}m_{z}} \approx {\frac{s_{w}\rho\; C_{p}T_{m}}{0.483\left( {1 - r_{f}} \right)}.}} & \left( {{Formula}\mspace{14mu} 2.2} \right)\end{matrix}$

This may serve as a rough estimate or rule of thumb for laser welding.Some parameters have been described before; here the welding speed isv_(w), the weld seam width is s_(w), the melting point for this width isT_(m), and the reflectivity is r_(f). However, finding accurate andgenerally applicable models for laser cutting or welding is challengingbecause of the high number of hard-to-estimate but influential processparameters, as well as many nonlinear or individually different effects.Therefore the laser welding and cutting processes are treated as blackbox models. The present invention employs the mechanism of machinelearning to manage the ins and outs of laser welding and cutting withoutnecessarily modeling the inside.

Feature Extraction and Dimensionality Reduction

The present invention does not seek nor desire to achieve human-likebehavior in machines. However, the investigation of something likecognitive capabilities within production machines of artificial agentscapable of managing laser processing tasks may provide an applicationscenario for some of the most sophisticated approaches towards cognitivearchitectures. Approaches for production machines may be structuredwithin a cognitive perception-action loop architecture, as shown in FIG.1C, which also defines cognitive technical systems. Cognitivecapabilities such as perception, learning, and gaining knowledge allow amachine to interact with an environment autonomously through sensors andactuators. Therefore, in the following, some methods known from machinelearning that will be suitable for different parts of a cognitiveperception-action loop working in a production system will be discussed.

If a cognitive technical system simply has a feature representation ofits sensor data input, it may be able to handle a higher volume of data.Moreover, extracting features emphasizes or increases thesignal-to-noise ratio by focusing on the more relevant information of adata set. However, there are many ways of extracting relevant featuresfrom a data set, the theoretical aspects of which are summarized in thefollowing.

In order to select or learn features in a cognitive way, we want to havea method that can be applied completely autonomously, with no need forhuman supervision. One way of achieving this is to use dimensionalityreduction (DR), where a data set X of size t×n is mapped onto a lowerdimension data set Y of size t×p. In this context

^(n) is referred to as observation space and

^(p) as feature space. The idea is to identify or learn a higherdimensional manifold in a specific data set by creating a representationwith a lower dimension.

Methods used to find features in a data set may be subdivided into twogroups, linear and nonlinear, as shown in FIG. 1D. Linear dimensionalityreduction techniques seem to be outperformed by nonlinear dimensionalityreduction when the data set has a nonlinear structure. This comes withthe cost that nonlinear techniques generally have longer execution timesthan linear techniques do. Furthermore, in contrast to nonlinear methodslinear techniques allow a straightforward approach of mapping back andforth. The question is whether a linear dimensionality reductiontechnique is sufficient for laser material processing, or if nonlineartechniques bring more advantages than costs. The following nonlineartechniques are very advantageous for artificial data sets: Hessian LLE,Laplacian Eigenmaps, Locally Linear Embedding (LLE), MultilayerAutoencoders (ANN Aut), Kernel PCA, Multidimensional Scaling (MDS),Isometric Feature Mapping (Isomap), and others. As a result Isomapproves to be one the best tested algorithms for artificial data sets. Wefind that the Isomap algorithm seems to be the most applicable nonlineardimensionality reduction technique for laser material processing.Therefore Isomap and two linear dimensionality reduction techniques areintroduced below.

Principal Component Analysis

Principal Component Analysis (PCA) enables the discovery of featuresthat separate a data set by variance. It identifies an independent setof features that represents as much variance as possible from a dataset, but are lower in dimension. PCA is known in other disciplines asthe Karhunen-Loève transform and the part referred as Singular ValueDecomposition (SVD) is also a well-known name. It is frequently used instatistical pattern or face recognition. In a nutshell, it computes thedominant eigenvectors and eigenvalues of the covariance of a data set.

We want to find a lower-dimensional representation Y with t×p elementsof a high-dimensional data set t×n mean adjusted matrix X, maintainingas much variance as possible and with decorrelated columns in order tocompute a low-dimensional data representation y_(i) for the data setx_(i). Therefore PCA seeks a linear mapping M_(PCA) of size n×p thatmaximizes the term tr(M_(PCA) ^(T)cov(X)M_(PCA)), with M_(PCA)^(T)M_(PCA)=I_(p) and cov(X) as the covariance matrix of X. By solvingthe eigenproblem withcov(X)M _(PCA) =M _(PCA)Λ,  (Formula 2.3)we obtain the p ordered principal eigenvalues with the diagonal matrixgiven by Λ=diag(λ₁, . . . , λ_(p)). The desired projection is given byY=XM _(PCA),  (Formula 2.4)gives us the desired projection onto the linear basis M_(PCA). It can beshown that the eigenvectors or principal components (PCs) that representthe variance within the high-dimensional data representation are givenby the p first columns of the matrix M_(PCA) sorted by variance. Thevalue of p is determined by analysis of the residual variance reflectingthe loss of information due to dimensionality reduction.

By finding an orthogonal linear combination of the variables with thelargest variance, PCA reduces the dimension of the data. PCA is a verypowerful tool for analyzing data sets. However, it may not always findthe best lower-dimensional representation, especially if the originaldata set has a nonlinear structure.

Linear Discriminant Analysis

Despite the usefulness of the PCA, the Linear Discriminant Analysis(LDA) may be seen as a supervised dimensionality reduction technique. Itcan be categorized as using a linear method, because it also gives alinear mapping M_(LDA) for a data set X to a lower-dimension matrix Y,as stated for M_(PCA) in equation 2.4. The necessary supervision is adisadvantage if the underlying desire is to create a completelyautonomous system. However, LDA supports an understanding of the natureof the sensor data because it can create features that represent adesired test data set.

Because the details of LDA and Fisher's discriminant are known, thefollowing is a brief simplified overview. Assume we have the zero meandata X. A supervision process provides the class information to divide Xinto C classes with zero mean data X_(c) for class c. We can computethis with

$\begin{matrix}{{S_{w} = {\sum\limits_{c = 1}^{C}{{cov}\left( X_{c} \right)}}},} & \left( {{Formula}\mspace{14mu} 2.5} \right)\end{matrix}$the within-class scatter S_(w), a measure for the variance of class cdata to its own mean. The between-class scatter S_(b) followsS _(b)=cov(X)−S _(w).  (Formula 2.6)

Between-class scatter is a measure of the variance of each classrelative to the means of the other classes. We obtain the linear mappingM_(LDA) by optimizing the ratio of the between-class and within-classscatter in the low-dimensional representation using the Fishercriterion,

$\begin{matrix}{{J(M)} = {\frac{M^{T}S_{b}M}{M^{T}S_{w}M}.}} & \left( {{Formula}\mspace{14mu} 2.7} \right)\end{matrix}$

Maximizing the Fisher criterion by solving the eigenproblem for S_(w)⁻¹S_(b) provides C−1 eigenvalues that are non-zero. Therefore, thisprocedure seeks the optimal features to separate the given classes in asubspace with linear projections.

LDA thus separates a low-dimensional representation with a maximizedratio of the variance between the classes to the variance within theclasses.

Isometric Feature Mapping

The PCA and LDA methods produce linear mapping from a high-dimensionaldata set to a low-dimensional representation. This may be expressed aslearning a manifold in an observation space and finding a representationfor this in a lower-dimensional feature space. For data sets with anonlinear structure, such as the artificial Swiss-roll data set, linearprojections will lose the nonlinear character of the original manifold.Linear projections are not able to reduce the dimension in a conciseway: data points in the feature space may appear nearby although theywere not in the observation space. In order to address this problem,nonlinear dimensionality reduction techniques have recently beenproposed relative to the linear techniques. However, it is a prioriunclear whether nonlinear techniques will in fact outperform establishedlinear techniques such as PCA and LDA for data from real industriallaser processing sensor systems, which will be investigated in theremainder of this thesis.

Isometric Feature Mapping or the Isomap algorithm attempts to preservethe pairwise geodesic or curvilinear distances between the data pointsin the observation space. In contrast to a Euclidean distance, which isthe ordinary or direct distance between two points that can be measuredwith a ruler or the Pythagorean theorem, the geodesic distance is thedistance between two points measured over the manifold in an observationspace. In other words, we do not take the shortest path, but have to useneighboring data points as hubs to hop in between the data points. Thegeodesic distance of the data points x_(i) in observation space may beestimated by constructing a neighborhood graph N that connects the datapoint with its K nearest neighbors in the data set X. A pairwisegeodesic distance matrix may be constructed with the Dijkstra's shortestpath algorithm. In order to reduce the dimensions and obtain a data setY, multidimensional scaling (MDS) may be applied to the pairwisegeodesic distance matrix. MDS seeks to retain the pairwise distancesbetween the data points as much as possible. The first step is applyinga stress function, such as the raw stress function given by

$\begin{matrix}{{{\Phi(Y)} = {\sum\limits_{ij}\left( {{{x_{i} - x_{j}}} - {{y_{i} - y_{j}}}} \right)^{2}}},} & \left( {{Formula}\mspace{14mu} 2.8} \right)\end{matrix}$in order to gain a measure for the quality or the error between thepairwise distances in the feature and observation spaces. Here,∥x_(i)−x_(j)∥ is the Euclidean distance of the data points x_(i) andx_(j) in the observation space with y_(i) and y_(j) being the same forthe feature space. The stress function can be minimized by solving theeigenproblem of the pairwise distance matrix.

The Isomap algorithm thus reduces the dimension by retaining thepairwise geodesic distance between the data points as much as possible.

Classification for Machine Learning

In machine learning, it is not only the extraction of features that isof great scientific interest, but also the necessity of taking decisionsand judging situations. Classification techniques may help a machine todifferentiate between complicated situations, such as those found inlaser processing. Therefore classifiers use so-called classes thatsegment the existing data. These classes can be learned from a certaintraining data set. In the ongoing research into AI and cognitivemachines, Artificial Neural Networks were developed relatively early inthe process. In comparison, the concepts of Kernel Machines andreinforcement learning appeared only recently but showed increasedcognitive capabilities.

Artificial Neural Networks

Artificial Neural Networks (ANN) have been discussed extensively fordecades. ANN was one of the first successes in the history of ArtificialIntelligence. Using natural brains as models, several artificial neuronsare connected in a network topology in such a way that an ANN can learnto approximate functions such as pattern recognition. The model allows aneuron to activate its output if a certain threshold is reached orexceeded. This may be modeled using a threshold function. Naturalneurons seem to “fire” with a binary threshold. However, it is alsopossible to use a sigmoid function,

$\begin{matrix}{{{f(x)} = \frac{1}{1 + {\mathbb{e}}^{- {vx}}}},} & \left( {{Formula}\mspace{14mu} 2.9} \right)\end{matrix}$with v as parameter of the transition. For every input connection, anadjustable weight factor w_(i) is defined, which enables the ANN torealize the so-called learning paradigm. A threshold function o can beexpressed using the weight factors W and the outputs from the precedingneurons P, o=W^(T) P, with a matrix-vector notation. The neurons can belayered in a feedforward structure, Multi-Layer Perceptron (MLP) or, forexample, with infinite input response achieved using feedback loops witha delay element in so-called Recurrent Neural Networks. A MLP is afeedforward network with a layered structure; several hidden layers canbe added if necessary to solve nonlinear problems. The MLP can be usedwith continuous threshold functions such as the sigmoid function inorder to support the backpropagation algorithm stated below forsupervised learning. This attempts to minimize the error E in

$\begin{matrix}{{E = {\frac{1}{2}{\sum\limits_{i}\left( {z_{i} - a_{i}} \right)^{2}}}},} & \left( {{Formula}\mspace{14mu} 2.10} \right)\end{matrix}$from the current output a_(i) of the designated output z_(i), where theparticular weights are adjusted recursively. For an MLP with one hiddenlayer, if h_(j) are hidden layer values, e_(i) are input values, α≧0 isthe learn rate, and ε_(i)=z_(i)−a_(i), then the weights of the hiddenlayer w_(ij) ¹ and the input layer w_(ij) ² are adjusted according to,

$\begin{matrix}{{{\Delta\; w_{ij}^{1}} = {{\alpha ɛ}_{i}h_{j}}},} & \left( {{Formula}\mspace{14mu} 2.11} \right) \\{{\Delta\; w_{ij}^{2}} = {\alpha{\sum\limits_{m}{e_{m}w_{mi}^{1}{e_{j}.}}}}} & \left( {{Formula}\mspace{14mu} 2.12} \right)\end{matrix}$

The layers are enumerated starting from the input to the output. Forbackpropagation, the weights are adjusted for the corresponding outputvectors until the overall error cannot be further reduced. Finally, fora classification of C classes, the output layer can consist of either Coutput neurons, representing the probability of the respective class, ora single output neuron that has defined ranges for each class.

ANN can thus learn from or adapt to a training data set and can find alinear or a nonlinear function from N input neurons to C output neurons.This may be used for classification to differentiate a set of classes ina data set.

Kernel Machines

In general, a classification technique should serve the purpose ofdetermining the probability of learned classes occurring based on themeasured data. Classification can be mathematically formulated as a setof classes c_(i)=c₁, . . . , c_(N) in C, with a data set represented byx_(i)ε

, and a probability of p_(i),p _(i) =p(c _(i) |x _(i))=f _(c)(x _(i),θ).  (Formula 2.13)

The parameter θ may then be chosen separately for every classificationor can be learned from a training data set.

In order to achieve learning, it is desirable to facilitate efficienttraining algorithms and represent complicated nonlinear functions.Kernel machines or Support Vector Machines (SVM) can help with bothgoals. A simple explanation of SVM, or in this particular contextSupport Vector Classification (SVC), is as follows: in order todifferentiate between two classes, good and bad, we need to draw a lineand point out which is which; since an item cannot be both, a binarydecision is necessary, c_(i)ε{−1, 1}. If we can only find a nonlinearseparator for the two classes in low-dimensional space, we can find alinear representation for it in a higher-dimensional space, ahyperplane. In other words, if a linear separator is not possible in theactual space, an increase of dimension allows linear separation. Forinstance, we can map with function F a two-dimensional space f₁=x₁,f₂=x₂ with a circular separator to a three-dimensional space f_(I)=x₁ ²,f_(II)−x₂ ², f_(III)=√{square root over (2)}x₁x₂ using a linearseparator, as illustrated in FIG. 1E.

SVC seeks for this case an optimal linear separator, a hyperplane,H={xε

³ |ox+b=0}  (Formula 2.14)in the corresponding high-dimensional space for a set of classes c_(i).In three-dimensional space, these can be separated with a hyperplane, H,where o is a normal vector of H, a perpendicular distance to the origin|b|/∥o∥, and o with an Euclidean norm of ∥o∥. In order to find thehyperplane that serves as an optimal linear separator, SVC maximizes themargin given by,

$\begin{matrix}{{{d\left( {o,{x_{i};b}} \right)} = \frac{{{ox}_{i} + b}}{o}},} & \left( {{Formula}\mspace{14mu} 2.15} \right)\end{matrix}$between the hyperplane and the closest data points x_(i). This may beachieved by minimizing the ratio ∥o∥²/2 and solving with the optimalLagrange multiplier parameter α_(i). In order to do this, theexpression,

$\begin{matrix}{{{\sum\limits_{i = 1}^{l}\alpha_{i}} + {\frac{1}{2}{\sum\limits_{j = 1}^{l}{\sum\limits_{k = 1}^{l}{\alpha_{i}\alpha_{j}c_{i}{c_{j}\left( {x_{i} \cdot x_{j}} \right)}}}}}},} & \left( {{Formula}\mspace{14mu} 2.16} \right)\end{matrix}$has to be maximized under the constraints α_(i)≧0 and Σ_(i)═_(i)c_(i)=0.The optimal linear separator for an unbiased hyperplane is then givenusing,

$\begin{matrix}{{{f(x)} = {{sign}\left( {\sum\limits_{i}{\alpha_{i}{c_{i}\left( {x \cdot x_{i}} \right)}}} \right)}},} & \left( {{Formula}\mspace{14mu} 2.17} \right)\end{matrix}$allowing a two-class classification.

SVM has two important properties: it is efficient in computationalruntime and can be demonstrated with equations 2.16 and 2.17. First, theso-called support vectors or set of parameters α_(i) associated witheach data point is zero, except for the points closest to the separator.The effective number of parameters defining the hyperplane is usuallymuch less than l, increasing computational performance. Second, the dataenter expression 2.16 only in the form of dot products of pairs ofpoints. This allows the opportunity of applying the so-called kerneltrick withx _(i) ·x _(j)

F(x _(i))·F(x _(j))=K(x _(i) x _(j)),  (Formula 2.18)which often allows us to compute F(x_(i))·F(x_(j)) without the need ofknowing explicitly F. The kernel function K(x_(i), x_(j)) allowscalculation of the dot product to the pairs of input data in thecorresponding feature space directly. However, the kernel functionapplied throughout the present invention is the Gaussian Radial BasisFunction and has to fulfill certain conditions, as inK _(G)(x _(i) ,x _(j))=e ^(−γ∥x) ^(i) ^(−x) ^(j) ^(∥) ² ,  (Formula2.19)with γ as the adjustable kernel parameter.

Because we have so far discussed only binary decisions between twoclasses, we note here that it is also possible to enable soft andmulti-class decisions. The latter can be achieved in steps by a pairwisecoupling of each class c_(i) against the remaining n−1 classes.

SVC can thus be used to learn complicated data. It structures this datain a set of classes in a timely fashion. Mapping into ahigher-dimensional space and finding the optimal linear separatorenables SVM to use efficient computational techniques such as supportvectors and the kernel trick.

Fuzzy K-Nearest Neighbor

Unlike the previously discussed Support Vector Machines, a lesscomplicated but highly efficient algorithm called the Fuzzy K-NearestNeighbor (KNN) classifier can also separate classes within data. Thealgorithm can categorize unknown data by calculating the distance to aset of nearest neighbors.

Assume we have a set of n labeled samples with membership in a knowngroup of classes. If a new sample x arrives, it is possible to calculatemembership probability p_(i)(x) for a certain class with the vector'sdistance to the members of the existing classes. If the probability ofmembership in class A is 90% compared to class B with 6% and C with just4%, the best results seem to be apparent. In contrast, if theprobability for membership in class A is 45% and 43% for class B, it isno longer obvious. Therefore KNN provides the membership information asa function to the K nearest neighbors and their membership in thepossible classes. This may be summarized with

$\begin{matrix}{{{p_{i}(x)} = \frac{\sum\limits_{j}^{K}{p_{ij}\left( \frac{1}{{{x - x_{j}}}^{\frac{2}{m - 1}}} \right)}}{\sum\limits_{j}^{K}\frac{1}{{{x - x_{j}}}^{\frac{2}{m - 1}}}}},} & \left( {{Formula}\mspace{14mu} 2.20} \right)\end{matrix}$where p_(ij) is the membership probability in the ith class of the jthvector within the labeled sample set. The variable m is a weight for thedistance and its influence in contributing to the calculated membershipvalue.

When applied, we often set m=2 and the number of nearest neighbors K=20.

Reinforcement Learning

In contrast to previous learning methods, which learn functions orprobability models from training data, reinforcement learning (RL) canfacilitate learning using environmental feedback from an agent's ownactions in the long-term, without the need for a teacher. This entailsthe difference between supervised and unsupervised learning. If along-term goal is sought, positive environmental feedback, also known asreward or reinforcement, may support improvement. An agent may learnfrom rewards how to optimize its policy or strategy of interacting withthe real world, the best policy being one that optimizes the expectedtotal reward. RL does not require a complete prior model of theenvironment nor a full reward function. The artificial agents thereforeindicate cognitive capability and act in a similar manner to animals,which may learn from negative results like pain and hunger and frompositive rewards like pleasure and food. In this case we pick that theagent has to use a value function approach, in which it attempts tomaximize its environmental return.

In RL, an agent takes actions, a_(t), in an environment that itperceives to be its current state, s_(t), in order to maximize long-termrewards, r_(t), by learning a certain policy, π. However, before we canstart learning with reinforcement we have to find answers regarding theappropriate agent design. The agent could try to maximize the expectedreturn by estimating the return for a policy π. This agent behavior isalso referred to as value function estimation. The agent may evaluatethe action by estimating the state value using a state-value functionV_(π)(s), considering a certain policy π_(w) that is continuouslydifferentiable, as in

$\begin{matrix}{{V_{\pi}(s)} = {{E\left( {{{\sum\limits_{t = 0}^{\infty}{\gamma^{t}r_{t}}}❘s_{0}} = s} \right)}.}} & \left( {{Formula}\mspace{14mu} 2.21} \right)\end{matrix}$

Using this function the agent may estimate the expected return for agiven state and a following policy. It could also estimate the expectedreturn for an action, following a given state and policy. Therefore, theagent chooses an action considering the given state from thestate-action function or Q-function, as in

$\begin{matrix}{{Q_{\pi}\left( {s,a} \right)} = {{E\left( {{{{\sum\limits_{t = 0}^{\infty}{\gamma^{t}r_{t}}}❘s_{0}} = s},{a_{0} = a}} \right)}.}} & \left( {{Formula}\mspace{14mu} 2.22} \right)\end{matrix}$

The next action therefore relies on a reward function r_(t) and in orderto allow the agent to grant a concession for expected future rewardsover current rewards, the discount factor 0≦γ≦1 may be selected. It ispossible to set how much the agent should discount for future rewards,for instance future rewards are irrelevant for γ=0.

In RL, the methods may be subdivided into groups such as value functionbased methods or direct policy search. Many different actor-criticalgorithms are value function based methods, estimating and optimizingthe expected return for a policy. In order to realize a value functionbased method, the behavior for an artificial agent and the underlyingcontrol problem may be stated as a Markov decision process (MDP). Thesystem perceives its environment over the continuous state set, wheres_(t)ε

^(k) and s₀ as the initial state. It can choose from a possible set ofactions a_(t)ε

^(m) in respect to a stochastic and parameterized policy defined asπ(a_(t)|s_(t))=p(a_(t)|s_(t), w_(t)), with the policy parameters wε

^(k). With a learned policy, it can be mapped from states to actionswith respect to the expected rewards r_(t)ε

. The reward after each action relies on r_(t)(s_(t), a_(t)). If noenvironmental model is available, the mentioned actor-critic methods canpotentially develop policy-finding algorithms. The name is derived fromthe theater, where an actor adapts its actions in response to feedbackfrom a critic. This can be obtained using a given evaluation function asa weighted function of a set of features or a so-called basis functionφ(s), which then gives the approximation of the state-value functionwith value function parameters v, as inV _(π)(s)=φ(s)^(T) v.  (Formula 2.23)

Improving the policy is an optimization issue that may be addressed witha policy gradient. The choice of the policy gradient method is criticalfor convergence and efficiency. Both seem to be met by the NaturalActor-Critic (NAC) algorithm, as described by J. Peters and S. Schaal,“Natural actor-critic”, Neurocomputing, Vol. 71, no 7-9, pp. 1180-1190,2008, where the actor improves using the critic's policy derivative g asin equation 2.24,g=∇ _(w) log π(a _(t) |s _(t)).  (Formula 2.24)

The steps for improvement of policy parameters of the NAC algorithm arethen calculated using,w _(t+1) =w _(t) +αĝ,  (Formula 2.25)where α is the learning rate, and ĝ is the natural gradient calculatedusing the Fisher metric or is derived from the policy as demonstratedwithin the mentioned NAC algorithm publication. The NAC algorithm withLSTD-Q is fully documented at table 1 on page 1183 of J. Peters and S.Schaal, “Natural actor-critic”, Neurocomputing, vol. 71, no. 7-9, pp.1180-1190, 2008. It is applied with a parameterized policy π(a|s)=p(a|s,w) initial parameters w=w₀ comprising the following steps in pseudocode:

1:  START: Draw initial state s₀ ~ p(s_(t)) and select parametersA_(t+1) = 0; b_(t+1) = z_(t+1) = 0 2:  For t = 0,1,2,...do 3:  Execute:Draw action a_(t)~ π(a_(t)|s_(t)), observe next state    s_(t+1) ~p(s_(t+1) | s_(t,) a_(t)), and reward r_(t) = r(s_(t,) a_(t)). 4: Critic Evaluation (LSTD-Q(λ)): Update 4.1:   basis functions: {tildeover (φ)}_(t) =[φ(s_(t+1))^(T) ,0^(T)]^(T) , {circumflex over (φ)}_(t)=[φ(s_(t))^(T) ,            ∇_(w) logπ(a_(t)|s_(t))^(T)]^(T) ,4.2:   statistics: z_(t+1) =λz_(t) + {circumflex over (φ)} _(t) ;A_(t+1) = A_(t) + z_(t+1) (φ _(t)− γ {tilde over (φ)}_(t))^(T) ;        b_(t+1) =b_(t) + z_(t+1) r_(t), 4.3:   critic parameters:[v_(t+1) ^(T) , ĝ_(t+1) ^(T)]^(T) =A_(t+1) ⁻¹ b_(t+1,) 5:  Actor: Ifgradient estimate is accurate, update policy parameters 5.1: w_(t+1) =w_(t) + α ĝ _(t+1) and forget (reset) statistics. END.

The basis functions φ(s) may be represented by mapping the sensor datainput into a feature space as we discussed it elsewhere in thisdocument. In this case the basis functions are equal to the featurevalues. The basis functions may as well be chosen differently or theagent may use raw sensor data. The basis function may as wellincorporate adaptive methods or an own learning step, that maximizeswith the reward function results.

It is important to note that other RL agents are applicable as well.Many other policy learning agent concepts may be applied. It furthermoreis inventive to use other sources as reward signal r_(t) besides theclassification output or quality indicator. For instance it is possibleto apply a post-process or pre-process sensor as reward signal source.If a camera-based or laser triangulation post-process sensor monitorsthe processing results and a user desires a specific weld seam width,such as 5 mm, the reward signal could give positive rewards whenever thedesired weld seam width is achieved and negative if it misses thedesired weld seam width. The reward function could be the probabilityvalue between 0 and 1 or −1 to 1 of a measured data of a post-processsensor to be part of a good or bad class, which is determined by aclassifier as described above. In case a pre-process sensor is used forgiving a reward r_(t), a measuring result like a crossing point of atriangulation line crossing the joint area of a workpiece or twoworkpieces, which usually results in a good processing result afterprocessing, could be used as a classification boundary forclassification and thus for a reward function. An RL agent could find aparameter set to achieve this goal. For industrial use cases this wouldserve as standalone system, without the necessity to teach aclassification unit. The RL agent would learn parameters to adjust theincoming feature values in order to achieve a specific weld seam width.Such as system may be delivered without the necessity to find anyparameters, the RL agent would choose them. The same would be possiblewith an RL agent learning to achieve a predefined meltpool size, kerfwidth, or cutting quality. The RL agent could learn from featuresgenerated from different sensor data sources such as photodiode data,camera sensors, acoustic sensor, processing gas values, etc. The RLagent could adapt laser power, x/y/z position/movement of the processinghead relative to the workpiece, processing gas type and pressure,feed-rate of added materials in case of other cladding, cutting,welding, soldering, or material processing techniques. Especiallynoteworthy is that the discussed techniques are applicable tolaser-hybrid welding, laser soldering, arc welding, plasma welding andcutting. Another exemplary welding setup would be to have in-processphotodiode sensors and a post-process triangulation sensor giving areward signal for an RL agent for a specific welding seam width. Anotherexemplary cutting setup would be to have in-process features from acamera or photodiodes for an RL agent learning how to control processinggas pressure. It is furthermore applicable to give the RL agents actionboundaries, limiting their range of actions but also increasing processstability.

Thus reinforcement learning may be a step towards a long-term goal inthat it entails learning a policy from given rewards usingpolicy-finding algorithms such as the Natural Actor-Critic.

Cognitive Capabilities for Production Workstations

Cognitive Technical Architecture

An artificial agent is anything that perceives its environment throughsensors and acts in consequence of this through actuators. An agent isdefined as an architecture with a program. The inspirational role modelfor this is natural cognition, and we want to realize a similar actingcognition for technical systems. Therefore, the agent will be equippedwith cognitive capabilities, such as abstracting information, learning,and decision making for a manufacturing workstation. As part of theprocess, this section introduces an architecture that creates andenables agents to manage production tasks. In order to do so, the agentsfollow a cognitive perception-action loop, by reading data from sensorsand defining actions for actuators.

A natural cognitive capability is the capacity to abstract relevantinformation from a greater set of data and to differentiate betweencategories within this information. Transferring this concept fromnatural cognition to the world of mathematical data analysis, acombination of data reduction techniques and classification methods isused according to the present invention to achieve something thatexhibits similar behavior. In industrial production, many manufacturingprocesses can be carried out using a black box model, focusing on theins and outs of the box rather on than what actually happens inside. Theconnections to the black box that may be used in production systems aregenerally sensors and actuators. Sensors such as cameras, microphones,tactile sensors, and others monitor the production processes. Thesesystems also need actuators, such as linear drives or roboticpositioning, in order to interact with its environment. For everyproduction process, these actuators have to be parameterized. In orderto learn how an agent can adaptively control at least one parameter ofthese production systems, many combinations of self-learning algorithms,classification techniques, knowledge repositories, feature extractionmethods, dimensionality reduction techniques, and manifold learningtechniques could be used. The present invention provides also differentcontrolling techniques, both open- and closed-loop, using multipledifferent sensors and actuators. After many simulations and experiments,a simple architecture that demonstrates how these techniques may becombined proved to be successful and reliable, at least for lasermaterial processing. However, the laser processes may be interpreted asa form of black box, and may thus be applicable to other types ofproduction processes.

FIG. 2A illustrates a cognitive architecture that may be suitable fordesigning agents that can provide monitoring or adaptive process controlfor production tasks. The diagram describes the unit communication andinformation processing steps. Natural cognition seems to abstractinformation firstly by identifying representative symbolism, such asstructured signals. A similar process can be accomplished usingdimensionality reduction (DR), in which the agent uses a low-dimensionalrepresentation of the incoming sensor data. Natural cognition thenrecognizes whether or not knowledge about the incoming sensationalevents is already present. This step may be achieved by usingclassification techniques that categorize “sensorial” events orcharacteristics. A natural subject may decide to learn or to plan newactions. In order to replicate this, the architecture of the presentinvention offers self-learning techniques that feeds a processing logic.In seeking to achieve quick reactions without the need to start acomplex decision-making process, we may also “hardwire” a sensor inputthat can directly initiate an actuator in using a closed-loop controldesign. Therefore, the architecture of the present invention may bedesigned in respect to four modes of usage, which will be discussedindividually in the following: first, abstracting relevant information;second, receiving feedback from a human expert on how to monitor andcontrol processes, or supervised learning; third, acting on learnedknowledge; and fourth, autonomously controlling processes in previouslyunknown situations.

A typical procedure used in production systems is to begin byconfiguring an assembly line, and then monitoring this for qualityassurance. This is also the case in laser material processing. Whenmaterials are processed using laser light, a high degree of precision isnecessary. However, welding or cutting with laser beams is difficult toobserve because of the strong radiation and process emissions. For thesereasons, many different sensors are used to monitor activities. Eventhen, it remains difficult for human experts to ascertain whether awelding action was successful or not by evaluating the monitoringresults. In industrial production, these processes are usually initiallyconfigured over manual trials, resulting in costs in labor andmachinery. All of the process parameters are kept constant because anychange would result in recalibration costs and may cause production tostop. A cognitive system of the present invention for laser materialprocessing that is capable of reacting appropriately to changes istherefore of great help and is an economic benefit.

As with other cognitive architectures in different contexts than lasermaterial processing, the aim here is similar, creating agents with somekind of artificial intelligence or cognitive capabilities related tohumans. Here the goal is to monitor or control processes inmanufacturing, where the adaptability of these techniques is anadvantage in creating agents for individual processes. When applyingthese solutions to production processes, the following requirements haveto be met. The components should be well-established and understood;they must only need limited configuration efforts, and should ideallywork ‘out-of-the-box’, capable of working with raw sensor data forinstance. In addition, they should be able to act quickly, in otherwords in real-time, with a repetition rate close to perfection or a lowerror rate. For example, the real-time requirement for laser materialprocessing means that a control-cycle has finished a completerun-through before the processing spot has left the initial position.For common processing speeds, this involves a minimum frequency of 500Hz.

One approach for a cognitive system design or for creating laserprocessing agents following the architecture introduced is shown in FIG.2B. Data processing is structured within this architecture, allowingcomparison of different agents. The agents may be composed of severalcomponents from different dimensionality reduction and classificationtechniques, which allow us to compare the performance of composed agentsand modules in terms of overall material processing quality. Manydifferent dimensionality reduction and classification techniques may beapplicable, and some of these have been evaluated in the researchproject. The cognitive architecture of the present invention offers thefollowing modules for composing agents: Principal Component Analysis(PCA), Linear Discriminant Analysis (LDA), Isometric Feature Mapping(Isomap), Support Vector Machines (SVM), Fuzzy K-Nearest Neighbors(KNN), Artificial Neural Networks (ANN), and reinforcement learning(RL), along with some other methods. Three embodiments of the presentinvention of control agents within this architecture would be agent Aconnecting Isomap, SVM, ANN, and PID welding control of the laser power,or agent B connecting Isomap, SVM, and PID laser cutting control of theprocessing gas, or agent C connecting ANN and Fuzzy KNN, for control ofthe z-axis.

As shown in FIG. 2D, the following sensors are used in the lasermaterial processing head 300 of the present invention: a high-speedcamera, sensors for solid-borne 302 and air-borne acoustics 304, andthree photodiodes 306 recording process emissions on differentwavelengths. Different kinds of coaxially integrated cameras 308 withCMOS sensors could be used according to the present invention.Recordings are preferably taken in the range of 200 frames per second(fps) to 1,500 fps and exposure times in the order of 4 ms and less,with region of interest sizes of 128×256 and 248×410 pixels and similar.An additional low-powered laser or a set of LEDs served as the optionalillumination unit for improved image quality. Three different wavelengthfiltered photodiodes that were integrated into the processing opticscaptured process emissions at 1,030-1,080 nm, at 1,150-1,800 nm, and at350-625 nm. The first bandwidth relates to laser back reflection; thelatter two correspond to temperature and metal-vapor or plasmaradiation. The data from the photodiode sensors 306 was recorded at asample rate of 20 kHz and 16 bit. As solid-borne acoustic sensors 302served two identical piezoelectric transducers mounted at two differentpositions on a workpiece 310. The data from the solid-borne acousticsensors 302 was recorded at a sample rate of 2 MHz or 192 kHz and 12 bitresolution. In some experiments, two microphones are used as air-borneacoustic sensors 304, one with an omni-directional and the other with aselected directional characteristic. One microphone was pointed towardsthe interaction zone, while the other captured the ambient sound. Thedata from air-borne acoustic sensors 304 was recorded at a sample rateof 96 kHz and 24 bit. The sensor alignment during a laser weld and anilluminated in-process camera picture is presented in FIG. 2C. The datafrom the sensors 302 to 308 are processed in a cognitive data processingunit 311.

As actuators, we have used the laser power and the processing gas. Thelaser source 312 that is used is a fiber laser, wherein the laser beamis coupled into the laser processing head 300 via a fiber 314. Thelasing wavelength is 1,070 nm in a continuous waveform, with a laserpower range of 50 to 1,500 W. The processing gas is pure nitrogen N₂. Asmounting and moving devices, we either used a six-axis robot 316 movingthe laser processing head 300 over the static workpiece 310, or movedthe workpiece under a static processing head using a transport unit. Theapplied optics are either YW50/52/30 processing heads for welding or aYRC processing head for cutting. It is emphasized that the lasermaterial processing head 300 could be employed as a laser cutting head100 or as a laser welding head 200, as shown in FIGS. 1A and 1B.

The sensor setup of the present invention allows to collect a lot ofsensor data from laser cutting or welding processes and to influencethis using critical process parameters through the selected actuators.According to the present invention, the high volume sensor data is thenreduced to relevant process information.

Abstract Relevant Information

In natural human cognition, we abstract or absorb information fromeverything that we hear, feel, and see. Therefore, we only generallyremember the most interesting things. Inspired by this, a technicalcognitive system should similarly abstract relevant information from aproduction process. Working with abstracted features rather than withraw sensor data has certain advantages. Many weak sensor signals may bereduced in dimension to fewer but better signals, resulting in a morereliable feature. Additionally, in order to realize real-time processcontrol, it is necessary to reduce the volume of the incoming sensordata because a greater amount of data may have a significant influencein causing longer execution times for the entire system.

The architecture of the present invention requires a test run in orderto abstract initial information. During this period of agent training,the parameter range of the actuator that will be controlled is altered.In order to determine which information is most relevant, the agentshould explore its own range of actions. After the initial referencetest, the system analyzes the recorded sensor data in order to discoverrepresentative features. The agent may solve feature calculationsseparately for different kinds of sensors, but the sensory units shouldideally be trained to map the sensory input into the learned featurespace. Finding a useful representation of the feature space is criticalbecause the system will only be able to recognize or react to changes inthe feature values. For the cognitive laser material processing system,we introduced cameras, photodiodes, and sensors for solid-borne andair-borne sound, offering a wealth of valuable process information.

The purpose of the cognitive processing of the present invention is toprovide as much information as possible for the subsequent processingsteps. However, the raw sensor data contains repetitions, correlations,and interdependencies that may be neglected. Therefore, in order toabstract the relevant information, the most significant features, orthose that contain the most information, should be identified. In orderto do this “cognitively”, an agent should perform this task without thenecessary supervision of a human expert. Therefore, a method of featureextraction is chosen that can be applied to all of the different kindsof processing tasks and the corresponding sensor data without the needto change parameterization or re-configuration. Manifold learning ordimensionality reduction techniques satisfy this need. They can reduce asensor data set X of dimension n in observation space to a data set Y ofdimension p in feature space. Often, the new quantity p is much lessthan n. However, many linear and nonlinear dimensionality reductiontechniques have been tried and tested for different purposes. Thepresent invention provides a suitable feature extraction technique forproduction workstations, complying with the following requirements thefeature extraction method works transparently and is able to display theprocessing steps to the user. The feature extraction method is able torun unsupervised. The feature extraction method is executable within areasonable time-frame for configuration, especially during processing.The extracted features contain enough process information for reliableclassification within several workpieces.

In essence, PCA seeks orthogonal linear combinations that represent agreater data set. These may be calculated for incoming sensor datavectors. Exemplary video data and its principal components aredemonstrated in FIG. 2E. These eigenvectors may serve as features forclassification up to a threshold d. Feature extraction combined withclassification may be achieved using Linear Discriminant Analysis.Analyzing the same data set using LDA and three learned quality classesdefined as “good”, “medium”, and “bad” provides another set of features,as is also demonstrated in FIG. 2E. Feature extraction may also beachieved using the Isomap algorithm. Unfortunately, the nonlinearfeature cannot be displayed in the same way as the linear featureextraction of LDA and PCA. All features show the importantcharacteristics from laser welding, such as keyhole, melt pool form andsize, as well as weld seam width. If an agent perceives a weldingprocess with these features, it may detect the named characteristics byprocessing just a few feature value signals, compared to thousands ofnoisy picture pixel signals.

The extracted features of the methods named above are compared in thefollowing. The LDA feature seems to contain more details than any one ofthe PCA features. Using this method of calculating, the LDA featuresseem to contain more process information in fewer features than PCAbecause they are especially designed to separate the desired classes.Furthermore, it is possible to display the calculated features using PCAand LDA in a way that makes these two methods more transparent thanIsomap. The user gets an idea of what a process looked like if a featureis identified in a process video simply by looking at it. PCA and Isomaphave the advantage that they can run unsupervised, which is not possiblewith LDA. Therefore, LDA merely serves as a comparison to PCA, but isnot considered as an alternative for the desired architecture.Furthermore, the LDA feature seems to be very individualized for aparticular process. Isomap has considerably higher execution times foranalysis and out-of-sample extension. Therefore, if classification withPCA achieves sufficient results, then it is more applicable to thesystem under research. Therefore, the method of choice would be PCA,unless Isomap shows a significantly better performance toward the firstobject of the present invention. We have to postpone the final choice ofdimensionality reduction techniques because the most important qualitymeasures are the experimental results, which are the basis of thepresent invention.

In essence, dimensionality reduction may allow agents to abstractrelevant information in terms of detecting variances and similaritiesduring a training trial. This helps the agent to process only a fewfeature values compared to the significantly higher volume of raw sensordata. Furthermore, dimensionality reduction may support the perceptionof similarities in unknown situations, for instance similar weldingcharacteristics such as melt pool size and form, even if these have notbeen part of the training. This may improve the adaptability of theagents to unknown but similar situations.

Supervised Learning from Human Experts

In natural human cognition, for instance in childhood, we often learnfrom others how to manage complex tasks. Similarly, a machine shouldhave the possibility of learning its task initially from a human expert.Supervised learning seems to be the most efficient way of setting up acognitive agent for production. In industrial production, a qualifiedhuman supervisor is usually present when the production system is beinginstalled or configured. The architecture that we are examining useshuman-machine communication in order to receive feedback from an expert,for instance through an intuitive graphical user interface on atouch-screen tablet computer. As mentioned above, at least one testaction per actuator or test run is needed in this architecture as aninitial learning phase. During these tests, the agent executes oneactuator from within the desired range of actions, and the sensor datainput is stored. After this run, an expert provides feedback concerningwhether the robot has executed the actuator correctly, or if the actionwas unsuccessful or undesirable. The feedback may come in many differentcategories so that different kinds of failures and exit strategies maybe defined. A classification technique may then collect the featurestogether with the corresponding supervisory feedback. Combined withlookup tables, the classifier module will serve as knowledge and as aplanning repository for a classification of the current system state.How an agent may perform its own actions and give itself feedback willbe of importance for the next section; this section mainly covers thecognitive capability of learning from a human expert, and theapplication of this knowledge for monitoring purposes.

Support Vector Machines, Fuzzy K-Nearest Neighbor, and Artificial NeuralNetworks as classification techniques will now be discussed. The morethat the human expert teaches the machine, the likelier it is that thesystem will achieve the desired goal. In order to save costs, thenecessary human supervisor time should be minimized to just one or tworeference tests, if possible.

Laser material processing systems as well as their processes are usuallyset up and configured by human experts. The architecture discussed maysimplify and accelerate this process. When the system performs a testaction, such as a laser power ramp for welding, a human expert indicateshow the processing result would be classified for the differentworkpiece areas using a graphical user interface that displays theworkpiece. For instance, the expert may mark a poor or medium weld thatdid not use enough laser power, a good weld, and a poor or medium weldthat had too much laser power applied. The system of the presentinvention stores this information together with the extractedcharacteristics or features described above using a classificationtechnique. All of the above named classifiers achieved good results; theextracted characteristics seem to be separable from each other for manydifferent process setups. In order to compare the performance of thedifferent classification techniques, the following quality measures maybe stated: The classification techniques should be executable within areasonable time-frame, especially if applied for closed-loop control.The classification should not have false positives and should be robustin transitional areas. The classification techniques should betransparent to the user. The classification techniques should makereasonable decisions in unknown situations.

A reasonable timeframe, or real-time for closed-loop control purposes,should not exceed 2 ms per cycle. The SVM, ANN, and Fuzzy KNN areclassification techniques which may all be used for classificationwithin a reasonable timeframe; however, KNN seems to be the fastest ofthese. It may be shown that the classification techniques have a verygood repetition rate in classifying data or features from laserprocessing, especially not having false positives in a high number ofexperiments. However, reliability within cognitive architecture alsodepends heavily on feature quality and the initial human expertfeedback. The detailed experimental evaluation in the following showsthe robustness and classification quality regarding the monitoring of alack of fusion when welding zinc-coated steel. However, inspection hasalready shown that ANN may not be as transparent as SVM and KNN. WithSVM, for instance, data clouds stored within the classifier may bevisualized, as is shown in FIG. 1E. Data stored in the many nodes of anANN does not seem to be as comprehensible for the user. However, ANNshowed very reasonable behavior in unknown situations in monitoringexperiments compared to the other techniques.

An ANN could be trained to classify N classes. This means that theoutput layer can consist of either C output neurons, representing theprobability of the respective class, or a single output neuron that hasdefined ranges for each class. The latter is used in this case for themonitoring experiments; the output neuron has a value of 0.0 for class1, 0.50 for class 2, 1.0 for class 3, and 0.25 or 0.75 where noclassification is possible. A single output neuron may not only servefor monitoring, it may also be a stable input signal for controllerequation 3.2. In this case, an MLP with 35 input neurons and two hiddenlayers proved to be suitable. The selected ANN configuration,“35-20-3-1” has been found in experiments as the best trade-off betweenaccuracy and the ability to generalize nonlinear welding processes. Theincoming sensor data stream was reduced in dimension using lineardimensionality reduction, as described above; the number of dimensionswas selected using residual variance analysis, resulting in 10 featuresfrom the diode sensors, 10 features from the solid-borne acousticsensors, 10 from the camera sensor, and 5 from the microphones. Theacoustic and photodiode sensor data has been processed for this caseusing fast Fourier transform and PCA.

The monitoring results are summarized in FIG. 2F. Once the ANN has beentrained with a laser power ramp from 50 W to 1,500 W, it is able tocategorize this kind of gradient when processing another workpiece withthe same laser power ramp. As a human expert taught the monitoring agentthat full workpiece penetration results in high connectivity, the agentnow provides a robust monitoring signal for this purpose. The agent ofthe present invention decides when the laser power is too low that thereis not enough connection, monitoring a signal value of 0.0, thusclass 1. The agent identifies when the correct laser power is appliedwith 0.50 for class 2, and provides the monitoring signal value 1.0 fortoo high laser power. Common issues in industrial welding are workpiecesthat are soiled with oil or grease that influence the welding result. Inthis case, the laser light can supposedly couple better into theworkpiece at the soiled areas than at a clean surface. The laser poweris therefore too high, resulting in a poor weld seam. Although the agenthas not been trained in this scenario, in this experiment the monitoringagent has been able to identify the soiled areas, with an appropriatesuggestion that the laser power is too high.

The classification method is an important module within the cognitivearchitecture which designs agents that are capable of providing reliablemonitoring signals. While KNN can be executed in the least time whilestill being transparent to the user, ANN and SVM have the ability todifferentiate complex data as it is the case for laser materialprocessing. Therefore, if KNN proves to be suitable in the experiments,it would be the method of first choice.

Semi-Supervised: Acting on Previously Learned Knowledge

The previous discussion shows how agents in the investigated cognitivearchitecture perceive their surroundings and learn from a human expert,as well as displaying their knowledge in terms of monitoring. Theprovided monitoring signal based on selected features is obtained fromdifferent sensors that are interpreted using a trained classifier. Thismonitoring signal seems to have improved quality and may be applicableto the control of process parameters. The agent would then change itsposition from observing the processing to actually acting upon thegained knowledge. However, if an agent is also applicable to processcontrol in industrial processing, it has to fulfill many requirementswith a performance close to perfection. The following are some of therequirements for the underlying cognitive architecture: The processcontrol module should be capable of completing at least onecontrol-cycle from sensor input to actuator output before theinteraction zone has moved on. The controlled parameter should have aneffect on the process outcome when altered, while simultaneouslyresponding in a timely fashion. The process control module should beoptimized in terms of providing a balance of reliable stability andnecessary dynamics.

In order to realize a robust process control that is suitable forindustrial production processes, a fast or real-time closed-loop controlis often required. An embodiment of real-time closed-loop controlarchitecture of the present invention is illustrated in FIG. 3. Theadvantage of the architecture under investigation is that the use offeatures rather than raw sensor data permits faster completion ofcontrol-loops with a minimal loss of information. In this architecture,any kind of controller design may be implemented that fits with theclassification output. A simple version would have three possibleclassification output values: too much, class I; correct, class II; andtoo little laser power, class III. This may be expressed using

$\begin{matrix}{{y_{e} = {\left\lbrack {- 101} \right\rbrack\left\lfloor \begin{matrix}\begin{matrix}p_{I} \\p_{II}\end{matrix} \\p_{III}\end{matrix} \right\rfloor}},} & \left( {{Formula}\mspace{14mu} 3.1} \right)\end{matrix}$where p are the class probabilities and y_(e) the quality indicator.

A PID controller could adjust a parameter of the system's actuatorsaccording to the monitoring signal discussed above concerning supervisedlearning from human experts. Combining PID-control with theclassification results enables the agents to perform laser powercontrolled processing. This can be realized as shown in

$\begin{matrix}{{c_{t} = {{Pe}_{t} + {I{\sum\limits_{i = {t - n}}^{t - 1}e_{i}}} + {D\left( {e_{t} - e_{t - 1}} \right)}}},} & \left( {{Formula}\mspace{14mu} 3.2} \right)\end{matrix}$with P for proportional, I for integral, and D for derivative behavior.The goal is to minimize the error e_(t) between the quality indicatory_(e), the output of the classification module, and the desired value of0.0. In this context, the inventive applicability of the desired valuein dependency of a probability class related quality indicator gives theopportunity to vary this value to optimize the desired process results.For instance, the laser processing system may learn how to weld with apenetration depth of 1 mm at a desired value of 0.0. Lowering thedesired value towards a value of −1.0 would result in less penetrationdepth. Increasing the same would result in higher penetration depth upto full or root penetration of a joint. Assuming the system would learnlaser power or processing gas parameters for a desired cutting qualityor kerf width, adjusting the desired value would result in wider orthinner kerf widths. Therefore, within this approach the system canestimate process models for individually different processes. Havingsuch a model represented by feature space mapping and classificationgives the user additional options to influence the process outcome. Theuser can for instance either fine tune single process outcomes workpiece by work piece or apply different desired values online whileprocessing the same work piece.

One approach describes a PID control with an ANN and correspondingexperiments. Another investigates the usage of an SVM classificationmodule to control laser welding. Other work uses processing gas as theactuator for a control agent in laser cutting in order to minimizedross. There is a description of the control of the z-axis by an ANNclassification module fed only by camera features. A comparison ofseveral control methods discusses SVM and ANN classification modules, aswell as bypassing classification and the linearized control ofindividual features. The latter method is also indicated to be a shortpath from dimensionality reduction to control in FIG. 2B. However, thecontroller module and its setup should be as simple as possible, with astandard configuration in order to maintain usability. If the processingtask requires it, more research into adaptive and nonlinear controllerdesigns as modules within the investigated architecture would presumablybe promising.

Unsupervised: Learn and Gain Knowledge from Agent Feedback

While a production system operates with a process control agent createdwithin the architecture discussed, it may be that the system experiencessomething new that it had not previously learned. Although every attemptwas made to keep all of the processing parameters constant for aconfigured process, influences may occur in varying workpieces, such aschanges in the mounting or workload properties. This may be the case inassembly lines if workloads change, or if any other process parameterthat is not recognized by the system is altered. The precision demandedin processes that treat metals with laser beams means that they aresensitive to the slightest, generally unintended change. A novelty checkbased on the trained data may detect such differences. In this cognitivearchitecture, this would result in a change of system mode, either tosupervised learning if a human expert were present, or to unsupervisedmachine learning. Thus, the cognitive agent may try to solve the problemby itself using a self-learning mechanism. In the remainder of thissection, a mapping of the characteristics is described as one proposedsolution to this problem.

Because it is inspired by natural cognition, the architecture of thepresent invention abstracts information, which reduces the volume ofdata. The term activation patterns may also be understood as featuresrepresenting sensory events. For instance, using the proposeddimensionality reduction module, a lower dimensional feature calculatedfrom the training events would indicate if the system has experienced acertain event. Presumably, it is more likely to identify similarities inunknown situations and trained data within the lower dimensional featurespace. Again, a classification method may be able to categorize anddistinguish all events. Because the proposed procedure of using thecognitive architecture involves training workpieces with an intendedactuator alteration, this may be utilized to map from one trainingworkpiece in a known process scenario to another in a different butsimilar surrounding, for instance when manufacturers change workloads.

As suggested, a self-learning mechanism is integrated into the system ofthe present invention. A novelty check on the basis of the trainedfeatures can detect new or previously unknown situations. In thesecases, the system performs another test action and classifies the newworkpiece using the previously trained features. This time, it does notneed to consult a human expert; it can map the gained knowledge onto thenew workpiece autonomously and can adjust the process controlappropriately. If this step is successful, the agent gains “fresh”features from the new process that may improve the ongoing productiontask. In simple words, the agent learns new features by mapping oldones. With this step, a workload change or sensor data offset, such as arise in temperature, could be overcome.

The following embodiment of the present invention describes the scenarioof changing material thickness in a cutting process, for instancebecause of a workload change. The new material is almost half as thick,with 0.7 mm stainless steel compared to 1.2 mm. The agent detects thatthe feature values are unknown during processing through a novelty checkwithin the classification unit. Because no human expert is present, theagent performs the same action as for the training workpieces. In thiscase, it alters its actuator, the laser power, from 1,500 W to 50 Wduring the training, just as it did in the last training. The agent thenmaps the features from the old workpiece by calculating theprobabilities from the old classes within the new workpiece, as is shownin FIG. 4. All of the previously trained areas have been identified inreasonable regions of the new workpiece; the agent excludes those areason the new workpiece that correspond to a kerf width to which the agentdoes not aspire. Because the agent knows the laser power that it hasapplied in this training, it can calculate the new features in thedetected class regions of the current workpiece. This expands thereaction probabilities of the agent to the new sensation capability,utilizing many different features that were both known before andrecently learned.

The cognitive architecture proposed for production systems and lasermaterial processing enables agents to gain several cognitivecapabilities, such as obtaining relevant information, learning from ahuman expert, and reacting to new situations based on previously learnedknowledge. This architecture may be used for different kinds of systemscontrolling one or several actuators based on the input of a high amountof sensor data. Compared with some other high-level learning approaches,the learning and reacting capabilities seem to be limited or nothuman-like; however, the architecture underlying this investigation hasthe potential to be very robust in terms of data acquisition. It is easyto use and can realize fast computing, up to real-time closed-loopcontrol of complex systems such as the industrial applications analyzedin the following.

Thus, cognitive capabilities: different agent designs enable twolearning modes: supervised and unsupervised. Demonstrations show thatthe agents can learn from a human expert and transfer knowledge forinstance how to cut a new workpiece almost just half as thick.

EXPERIMENTAL VALIDATION FOR LASER MATERIAL PROCESSING Introduction andDefinitions

If the offered cognitive architecture of the present invention isapplied to industrial laser material processing, there is improvement inindustrial laser welding, laser cutting, or other processes with similardemands. Firstly it is investigated whether welding defects such as alack of fusion may be learned and monitored. Then the possibility ofrelating knowledge, decision-taking utilizing classification techniques,and laser power control is examined. For unknown situations, areinforcement learning agent explores the possibilities and learnsparameter sets for laser welding.

The embodiments of the present invention have similar but differentconfigurations and parameters to those previously mentioned. However,all of the experimental setups are established in such a way that theyreflect common configurations that exist in industrial laser materialprocessing. Some notations and configurations are introduced in advanceregarding the optical setup, the materials used, common systemconfigurations, and the presentation of data. The importantconfigurations for the optical system are as follows: all embodiments ofthe present invention are carried out with fiber lasers at a maximumpower of either 1,500 W or 1,000 W, operating at a wavelength of 1,070nm. The processing optics are a YRC head for cutting and a YW52 head forwelding, with three photodiode sensors. Both processing heads areequipped with a coaxially integrated CMOS camera. The interaction zoneis illuminated using a de-focused, low cost laser. The focal spotposition during the welding processes was set on top the workpiecesurface. For welding, the fiber diameter is 100 μm, the focal spotdiameter is 200 μm, the Rayleigh-length is 2.6 mm, the focal length is250 mm, and collimation length is 125 mm. During the cutting process,the focal spot position was set at 1.5 mm beneath the workpiece surface.For cutting, the fiber diameter is 100 μm, the focal spot diameter is136 μm, the Rayleigh-length is 1.3 mm, the focal length is 100 mm, andcollimation length is 73 mm. The relative distance of the processinghead to the workpiece surface was maintained at 0.7 mm by a capacitivedistance controller. As mentioned previously, laser cutting refers tofusion cutting only. For cutting, the processing gas nitrogen was set at17 bar. During welding, nitrogen was also used as shielding gas. Theprocessing heads were mounted on a six-axis robot. The imprecision ofthis during actions is responsible for the occasional signaldisturbances in the graphs presented later. Different materials are usedfor the experiments, and these are defined using the DIN EN 10020European norm material number. For the remainder of this chapter,stainless steel refers to material no. 1.4301, mild steel refers tomaterial no. 1.0037, and zinc-coated steel refers to material no.1.0350. A few different standard sets of PID control parameters havebeen applied; for the remainder of this chapter, these values are P=10,I=0.5, D=0 for cutting, and P=0.5, I=0.1, D=0 for welding, unlessotherwise stated. In order to improve readability of the displayedgraphs, the data is smoothed over up to 0.04 s; while for improvedvisualization, the displayed camera features may slightly differ fromthose used to obtain the feature values curves. The standard systemclock rate of the data processing system, used sensors, and laser powercontrol is set at 1,000 Hz. The length of the processed workpieces is 30cm, with different thicknesses and processing speeds.

In the following, the first embodiment of the present inventionregarding monitoring gaps with a lack of fusion in laser welding shouldbe described.

Current industrial solutions may successfully monitor many defects thatoccur during joining processes. However, detecting insufficientconnection or a lack of fusion because of gaps between two sheets ischallenging. This defect often occurs if the gap between the two sheetsis too large. Even if the laser beam penetrates through the top andbottom layers of the two sheets, the gap may still be too large for asuccessful joint, and a complete lack of fusion or false friend canoccur. Because the beam has penetrated the top and bottom sheets, thedefect is often not visible when inspecting the workpiece afterprocessing.

Car manufacturers increasingly integrate zinc-coated alloys. Whenwelding zinc-coated workpieces, it is advantageous to leave a specificgap between the two work-pieces which will be joined. This gap allowsany zinc vapor to dissipate during processing. If the gap is too smallor does not exist, the welding process may suffer from spilings; if thegap is too large, the process may suffer from insufficient connection ora lack of fusion. The latter is hard to detect in post-monitoringbecause the weld seam may appear to be sufficient from the outside, evenif there is no connection at all. In-process monitoring has alsodifficulties to detect false friends. Photodiode sensors may not detecta lack of fusion, while coaxially-taken camera pictures are noisybecause of the comparatively high reactivity of zinc alloys. Even humanexperts find it hard to discern whether insufficient connection ispresent at the observed welding process from coaxially-taken camerapictures. Therefore, a quality aspect of a cognitive technical lasermaterial processing system would be that the monitoring agent can learnhow to detect complex welding faults such as lack of fusion in laserwelding. Therefore the cognitive architecture creates a monitoring agentcapable to learn. The agent's skills will be tested and dimensionalityreduction and classification results with a lap weld will be compared.

Agent Learning Mode

The monitoring agent of the present invention welds two workpieces ofzinc-coated steel in order to train itself. The learned feature valuegraphs, scans, and additional information of one of these workpieces isshown in FIG. 5. Afterwards, in the monitoring mode, the agent observesa set of workpieces and has to decide whether false friends are presentor not. The learning task is to extract features from the trainingworkpieces and to connect these with given human expert feedback. Thisfeedback marks the regions of insufficient connection within thetraining workpieces. Following this, the monitoring agent should be ableto detect faulty workpieces. In this embodiment, the agent uses threephotodiodes, a solid-borne acoustic sensor, and an in-process camerawith an illuminated interaction zone. In helping to detect falsefriends, the temperature photodiode and the in-process camera picturesprovided the best feedback. The sensor data from the video is reduced indimension using different dimensionality reduction techniques.

Exemplary camera pictures are demonstrated in the upper picture row ofFIG. 5. If a camera picture has a resolution of 100×80 pixels, the agentconverts this to a vector with 1×8,000 and combines it with the tincoming frames (i.e. t is the number of frames recorded in a certaintime window with a predetermined picture rate of the camera) to a datamatrix X of size t×n. This matrix then is reduced in dimension to matrixY of size t×p with p<n. For laser material processing, the sensor datacan generally be significantly reduced, with p<<n. The incoming sensordata contains approximately 30 MByte per second, which is reduced indimension. After this, the incoming feature set contains approximately10 kByte per second. In order to achieve the reduction, several methodsare applicable. The most promising methods, such as PCA, LDA, andIsomap, have been already described in detail. It is possible to displaythe eigenvectors of LDA and PCA features, as is shown in the middlepicture row and as a schematic illustration of the middle row in thelower picture row of FIG. 5. It has been surprisingly found that it issufficient to use the 10 most significant eigenvectors or features forfurther evaluation instead of calculating the residual variance. For thetraining regarding zinc-coated steel workpieces, the agent uses the twobest features from Isomap, the two best from PCA, and the best from LDA.The corresponding feature values vs. processing time are shown in FIG.5. The agent learns from human expert feedback that, in the indicatedarea class I, the connection is sufficient, and that there is a lack offusion in the indicated area class III. The area class II in FIG. 5 hasnot been given to the agent, although a lack of fusion is still present.The knowledge gained from the expert feedback is represented by a set ofclassifiers, which will be compared later. The indicated areas are usedto train the classifiers and LDA. The feature values in FIG. 5 showdifferent amplitudes in the regions with and without connection. In theregion with a lack of fusion (area class III), camera features 3 and 4have a high feature value amplitude. This means that the agent may usethese two features to differentiate whether or not a false friend ispresent or not.

Thus there is an advantage regarding cognitive capabilities: the systemof the present invention is adapted to abstract relevant information byreducing the incoming raw sensor data to a thousandth, still capable ofmonitoring.

This is a good point to compare the employed dimensionality reductiontechniques. Judging by the amplitudes of feature values or thesignal-to-noise ratio, the features extraction method may be orderedfrom better to worse by Isometric Feature Mapping (Isomap), LinearDiscriminant Analysis (LDA), and finally Principal Component Analysis(PCA) for a training workpiece. The eigenvectors from LDA and PCA may bereshaped to the original picture size, and this provides a betterunderstanding of what the features may indicate. In FIG. 5, thesefeatures are color mapped, with the area next to the area class IIindicating low variance, and the area class I and the area class IIIindicating contrary variance. Camera feature 3 from LDA shows a strongcorrelation with the presence of a lack of fusion. This means that thevariation within the in-process video between a lack of fusion andsufficient connection is described within this eigenvector. In thiscontext, camera feature 3 tells us that the false friend in this processcan be detected when there is a higher intensity in the camera featureareas class I and a lower intensity in the areas class III. Thisindicates that the presence of false friends is shown as a variation inthe weld pool form. This contributes as a comparison of dimensionalityreduction techniques: Isomap seems to be the best in terms ofunsupervised feature learning, while LDA is best for additional analysisthrough the inspection of eigenvectors.

It is remarkable that the weld seams shown in the scanned top and bottomsheets of the training workpiece in FIG. 5 show hardly any visibleindications from the outside of a lack of fusion between the joinedsheets. This again shows why it is so difficult to detect this defect,earning it the name false friend. However, the feature values of theeigenvectors from PCA, LDA and Isomap indicate a robust correlation tothe presence of a lack of fusion; the monitoring agent of the presentinvention has now been trained in this, and can apply this to detect thesame kind of defects in the next subsection.

Agent Monitoring Mode

The monitoring agent of the first embodiment of the present inventionhas to decide for the following workpieces whether a false friend ispresent or not based on its perception using features and learnedknowledge within its classifiers. The knowledge was generated when thehuman expert provided feedback during the learning mode. The classifiersin this agent use two categories: a “good” class I, with existingconnection; and a “bad” class III, with a lack of fusion. The classifieroutput or monitoring signal provides a calculated probability of whethera false friend is present. Two workpieces are monitored by this agent:workpiece Z001, as shown in FIG. 6; and workpiece Z002, as shown in FIG.7, with two inserted gaps of 1.0 mm and 0.6 mm at different positions.The figures also contain the false friend probability vs. the processingtime.

In order to calculate the false friend probability, the agent uses theclassification techniques Fuzzy K-Nearest Neighbor (KNN), ArtificialNeural Networks (ANN), and Support Vector Classification (SVC), whichhave been already described before in detail. The classifierconfiguration is set to the nearest neighbors for Fuzzy KNN, an ANNconfiguration of “6-20-3-1”, while SVM uses an RBF Kernel described inequation 2.19. All three of these used the same training data in orderto provide a comparison of their classification quality. The classifiersare taught the areas of the training workpieces and the correspondingfeature values. The task of each classification technique is to identifysimilarities between the high-dimensional observation space of incomingprocess features and the learned feature values. The classificationtechnique then calculates a probability for the presence of a lack offusion.

Workpieces Z001 and Z002 show little indication on the top or bottomsheets of a lack of fusion, as is shown in the scans in FIGS. 6 and 7.However, all of the tested classifiers agree that there are two largeareas of a lack of fusion per workpiece. The areas class I and III showhuman expert feedback for the tested workpieces. Generally speaking, ifthe classification threshold is 50%, then the monitoring agent agreeswith the human expert, except for the beginning of workpiece Z002. TheANN classifier falsely calculates the lack of fusion probability atbelow 50% in the first milliseconds of the processing time. However,despite this misjudgment by ANN, the other classifiers provideconservative estimates of a lack of fusion, even in the transitionalarea II indicated regions of the workpieces. Therefore, for the testedworkpieces Z001 and Z002, a monitoring agent with SVM or Fuzzy KNNclassification would have successfully detected the lack of fusionareas.

Comparing the three classification techniques, ANN did not detect a lackof fusion in a time frame smaller than 1% of the total processing time.SVM and Fuzzy KNN proved to be more robust in this trial. Furthermore,Fuzzy KNN showed more fluctuations than ANN and SVM, but also had thefastest processing times. However, all of the classification techniquesdisplayed comprehensible behavior for this classification task. It maybe that the differences between the classification methods are greaterwhen feature acquisition is worse.

Thus, a second advantage regarding monitoring will be apparent: Thesystem has achieved a successful detection of false friends thatoccurred because of inserted gaps of 1.0 and 0.6 mm within zinc-coatedsteel lap welds. A reliable detection may increase quality in car bodyproduction.

The monitoring agent has learned features on how to monitor theconnection from a human expert on two training workpieces. It has thendetected robustly the welding defect of a lack of fusion in the testedsamples. All of the classification methods discussed seem to beapplicable for use as the monitoring agent. However, the classificationprobability seems to provide a one-dimensional monitoring signal fordetecting classes of the learned features. Will it also be possible toapply this to controlling tasks?

Cognitive Closed-Loop Control of Laser Power in Cutting and Welding

In laser cutting and welding, as well as being of commercial interest,research has sought to create an adaptive system for controllingprocesses. Once reliable process control is achieved, the quality ofworkpieces may be improved, and efficiency and savings in thesignificant labor costs and environmental resources may be achieved.Having reliable process control would also strengthen the benefits ofusing lasers compared to other welding and cutting techniques, since itwould target the high configuration and manual trial efforts that arenecessary to maintain the required precision standards in laser materialprocessing. According to the present invention, cognitive capabilitiessuch as learning and decision making help to approach this goal. Oftenmany previous attempts suffered from the noisy sensor data input. Theabsence of global models or absolute sensor data values that areapplicable to a wide range of different welding or cutting processesseems to push the vision of online process control far into the future.However, a cognitive agent that can learn these different processeshelps to bypass this issue through being able to learn and adapt toindividually different processing tasks.

In order to achieve process feedback control, the monitoring signaly_(e) is used as the control variable. As actuating variable, whichcould possibly be any alterable process parameter with interrelationshipto y_(e), the laser power seems suitable for its low inertia and itsstrong relation to y_(e). Its magnitude is calculated by the PIDalgorithm as shown in equation 3.2. In order to achieve process control,the agent closes the loop by connecting the monitoring signal to a PIDcontroller, as is shown in equation 3.2. The feedback controller isdesigned as a single-input-single-output (SISO) control system, whichreceives the monitoring signal y_(e) from the classification unit, with0<y_(e)≦1 for too low and −1≦y_(e)<0 for too high laser power, and usesthis as reference value to minimize controller error. Variations oflaser power have a significant influence on the results in welding andin cutting processes and have a short response time, often less than 1ms. Another possibility would be to vary the processing speed, but theresponse time and precision of current robotic or carrier devices isworse when compared to laser power variation. Of course, variation inboth speed and laser power should be interdependent. If the welding orcutting results stay the same, the energy per unit length should remainapproximately the same, as stated in equations 2.1 and 2.2. Therefore,if the velocity rises, the laser power should be increased withapproximately linear correlation. However, because laser materialprocessing has many nonlinear effects, it is unfortunately not thatsimple. Referring to the equal energy per unit length level is a goodmethod for approximating, if a controller works comprehensibly.Therefore, velocity alteration is a convenient way of proving successfullaser power control, which will now be described in the remainder ofthis section for common industrial welding and cutting setups.

In the following, the second embodiment of the present inventionregarding agent control of laser cutting should be described.

The agent of a second embodiment of the present invention requires atleast one training workpiece with some laser power variation, in thiscase from 1,500 W to 50 W. The training workpiece is processed at 3.0m/min, as shown in FIG. 8. A human expert teaches the cutting controlagent to keep the cutting kerf stable at 0.4 mm by indicating thecorresponding region on the training workpiece. The kerf width variesduring cutting with alterations in laser power or processing velocity.Once the agent has trained itself, it will test its cognitivecapabilities by processing at different velocity levels. Furthermore,the agent has to transfer the learned knowledge to unknown situations,such as processing different materials or layers of multiple sheets.

The cutting control agent of the present invention applied throughoutall the experiments has trained itself with a stainless steel workpieceof 1.0 mm material thickness at a processing speed of 3.0 m/min. As isdemonstrated in FIG. 8, the learned features change for variations inthe applied laser power ramp, which enables reliable classification forfuture workpieces. The most significant feature amplitude change is theevent when the cut breaks, as occurs at 4 s of processing time. Thein-process camera picture at 4.2 s in the upper picture row in FIG. 8shows an intensity increase in the center of the picture when comparedto the earlier picture at 3 s. This difference is also incorporated intocamera features 3 and 4. The kerf width difference is incorporated asthe intensity variation in several of the displayed camera features.With this knowledge, the cutting agent attempts to control the followingworkpieces, to maintain the cutting quality, and to prevent a loss ofcut robustly.

The trained cutting control agent processes several workpieces atdifferent velocities, as is demonstrated in FIG. 9. Workpiece CA001 isprocessed at a velocity of 1.8 m/min, which is just 60% of theprocessing speed that the agent has been trained for. The agentconverges at an averaged laser power level of 729 W. The stretched kerfscans are shown in FIG. 10, highlighting reliable cutting quality withmaintained kerf width and no loss of cut. For the following workpieces,the processing speed is increased to 3.0 m/min for workpiece CA002, 4.8m/min for workpiece CA003, 6.0 m/min for workpiece CA004, 6.6 m/min forworkpiece CA005, and 7.8 m/min for workpiece CA006. The graph in FIG. 9shows that the applied laser power has been raised with increasingvelocity. This would be expected if the same energy per unit lengthlevel approximately produces a similar cutting quality. The scannedkerfs in FIG. 10 also show a maintained kerf width, as desired. Anotherset of scanned workpieces from CC002 to CC006, with the same cuttingspeed levels as the controlled ones but a constant laser power of 750 W,similar to the laser power that the agent had applied at CA001, is shownin FIG. 10. The kerf width is increasingly thin as the processing speedgets faster. For workpieces CC004, CC005, and CC006, the scans show thatthere is a loss of cut. Workpiece CC002, with a processing speed of 3.0m/min and constant laser power, shows a slight variation in kerf widthover the time of processing. This indicates that the tested cuttingprocess has nonlinear characteristics. This may be the reason why theagent first increases the laser power significantly, and later convergesat a lower power level during the processing. Next to effects at thestart of cutting, the early rise in laser power may result from amounting robot imprecision when the robot starts accelerating. Thisimprecision may also be observed in curve-shaped cutting lines at thestart of some of the processes shown in FIG. 10. However, for the testedworkpieces, an improvement in cutting quality can be observed during theconstant laser power trials when compared to those controlled by thetrained cutting agent.

It is not a common practice to switch materials in industrial productionand yet continue processing with the same system parameters because thecutting quality would differ significantly. It has been found that thetrained cutting agent of the present invention manages to showcomprehensible behavior in such a situation. In order to challenge thecognitive capabilities of the cutting agent further, the experimentdemonstrated in FIG. 11 shows its reactions for materials for which ithas not been trained. As the agent had not been trained for differentvelocities, but demonstrated reasonable behavior, it is now being testedfor different kinds of materials. The processing gas and all otherexperimental setup properties remain constant, therefore only thematerial is altered. The change of material results in the input of verydifferent sensor data. However, the perception of the cutting agentrelies on features that may help it to identify similarities better thansimple raw sensor data. The in-process camera pictures in FIG. 11demonstrate that raw sensor data differs greatly for differentmaterials.

Irrespective of the differences in the raw sensor data, the cuttingagent that has been trained for stainless steel shows robust convergencewhen processing zinc-coated steel or mild steel. None of the threeworkpieces have a loss of cut. However, the applied laser power for mildsteel seems to be higher than necessary. The cutting agent achieved thedesired goal of maintaining a minimum kerf width, yet its behavior maybe optimized for a change in material in terms of power consumption forthe tested workpieces.

Another possible alteration is to increase material thickness in orderto test the capability of the agent to process workpieces based onlearned features. Although again this is not a common practice inindustrial production, the material thickness may be increased byoverlapping several workpieces. This would provide another test toascertain whether the agent can comprehend the need to increase laserpower given the number of workpieces that it has underneath theprocessing head. Workpiece J001, shown in FIG. 12, consists of threesheets of stainless steel, each with a material thickness of 0.6 mm. Thecapacitive distance sensor maintains a constant space between theprocessing head and the workpiece surface of 0.7 mm. Furthermore, thePID values have been set to P=6.7, I=0.5, D=0 in order to improveperformance with large steps within the reference signal. The workpiecethus varies in thickness in two steps through approximately 0.6 mm, 1.2mm, to 1.8 mm. The applied power varies accordingly from an averaged 570W, 915 W, to 1,476 W. These values seem to be comprehensible, since theagent increases the applied laser power by approximately 500 W for eachadditional 0.6 mm sheet. However, when processing three sheets ofstainless steel at the same time, there is an increase in drossdevelopment at the bottom sheet. This may be caused by the fixed focalspot position and processing gas pressure, which are not optimized tocut three sheets at the same time.

Thus, an advantage with respect to adaptability will be apparent: theagents adapt to various situations, two different production processessuch as cutting and welding at different speeds; 50% less materialthickness in the bottom sheet in welding after additional training;three different materials with laser cutting such as zinc-coated steel,mild steel, and stainless steel.

For the tested workpieces, the task of preventing a loss of cut has beenachieved by the cutting agent irrespective of velocity, material, orthickness variations. Furthermore, the agent has maintained a certaincut quality in terms of a minimum kerf width in the experiments carriedout. In these experiments, the agent seems to decide intelligibly inunknown situations based more on process characteristics than on noisyraw sensor data.

It may be that some experts in laser material processing believe that itis more demanding to join something than to cut it. The question mayarise whether the cognitive capability could also be applied to laserwelding processes.

In the following, a third embodiment related to agent control of laserwelding will be described.

It is a common aim in industrial welding to join two parts with maximalconnection, yet to avoid excessive penetration without any weld rootconvexity or concavity on the obscured side of the workpiece surface. Aroot convexity or concavity occurs when the laser beam actually exitsthe workpiece on the bottom side, also known as full penetration weld.This leaves a noticeable convex or concave trace, which restricts afollowing paint job because the weld seam root would still be evident.As another manufacturing example, with pipe welding this type ofimperfection may cause undesired behavior in the fluid flow that cancause erosion or corrosion issues. If the laser power could becontrolled in terms of the penetration depth, it may be maintained atthe desired level, thus creating maximal connection without the beamleaving the workpiece. Due to the nonlinearity of the welding process,this goal is hard to achieve, and great efforts are spent on setting upwelding systems within this frame of operation. The following will showhow this task may be learned and handled by a cognitive agent.

Firstly, the agent requires a training workpiece, which it processeswith a laser power ramp from 1,000 W to 50 W. The recorded featurevalues are shown in FIG. 13. The human expert feedback is indicated inthe colored areas, corresponding to class 1 with too high laser power,class 2 with correct laser power, and class 3 with not enough laserpower. Please note that the bottom surface of the training workpieceshows a seam root convexity or in this case excessive penetrationbetween approximately 0 to 2.8 s of processing time. The feature valuesets differ greatly from each other between the selected classes, whichpresumably allows for reliable classification. The in-process picturestaken at different processing times show that the variance within thevideo pictures mainly relies on weld seam width, melt pool size andform, and the front of the heat affected zone. Furthermore, informationabout the keyhole seems to be present in slight intensity variationswithin this spot size, and yet this is not visible from the picturesshown. The agent abstracts these process characteristics by extractingthe learned camera features displayed in FIG. 13. These features aresensitive to variations in keyhole size, melt pool form or size, seamwidth, and other characteristics. Of course, the features from thephotodiodes are collected as well. The most prominent feature is thetemperature feature, which is also shown in FIG. 13. The system hasselected 10 features in total and has trained itself using thisworkpiece. For improved agent behavior, and because of the great numberof dimensions used in the feature space, a second training workpiece isintroduced to the agent to allow it access to more training samples, asis demonstrated in FIG. 13. This time, a slower processing speed isapplied in order to gain increased sensitivity by the agent to velocityalterations. Finally, with this knowledge, the agent should bewell-prepared to pursue the goal of maximum connection without rootconvexity in the following welding experiments.

The agent welds workpieces WA001-WA007 at different velocities, from 1.2m/min, 1.8 m/min, 3.0 m/min, 4.2 m/min, 5.4 m/min, 6.6 m/min, to 7.8m/min, as shown in FIGS. 15, 16, and 17. The agent decides to increasethe applied laser power accordingly with every additional velocity step.This seems to be the correct countermeasure, since the energy per unitlength level should stay approximately the same in order to achievesimilar welding results. FIG. 18 shows that the welds seem to have asimilar penetration depth at the different velocities. This is alsoindicated by the cross-sections that were analyzed at three points ofeach workpiece. The penetration depth is approximately 1.1 mm atdifferent locations and different velocities. This indicates that theagent seems to take reasonable decisions and to maintain a constantpenetration depth by controlling the laser power.

Another possible process variation in laser welding is a change inmaterial thickness. The agent should decrease the laser power when theworkpiece gets thinner. Therefore, the described welding agent will nowweld workpieces with 50% less material thickness in the bottom sheet.

FIG. 19 shows the first trial of welding a workpiece with 50% lessmaterial thicknesses in the bottom sheet than the welding agent has beentrained for. The training workpiece was stainless steel, with 0.6 mm atthe top sheet and 1.2 mm at the bottom sheet, while the currentworkpiece has 0.6 mm at the top sheet and 0.6 mm at the bottom sheet.Maintaining the same penetration depth as before is not possible forthis workpiece because the beam would exit the workpiece on the bottomside.

The scans of the workpiece surfaces in FIG. 19 indicate a root convexityat the bottom sheet. This means that the goal to weld the workpiece insuch a way that it can be painted later on has not been achieved becausethe agent picked too high a laser power for this workpiece. This agentnow needs human expert feedback in order to learn from its mistakes; anagent without further human expert feedback is investigated later. Here,the human expert has to teach the system again, providing feedback thatthe features for the marked areas in FIG. 19 had too high a laser powerapplied. With this feedback, the agent retrains its classification andperforms another weld, as shown in FIG. 20. The WA009 workpiece stillshows a region where the agent has not performed well. Therefore, thehuman expert provides another iteration of feedback to the agent,marking some regions with a minus sign (as indicated by the arrowsaccompanied by −) indicating too high a laser power and one region witha plus sign (as indicated by the arrow accompanied by +) for the correctlaser power. The next weld reaches the desired goal of achieving highconnection without root convexity, as can be seen in the scans for WA010in FIG. 17. Furthermore, the applied laser power on average is lowerthan for another workpiece processed with the same velocity but athicker bottom sheet. Therefore, the agent seems to demonstratereasonable behavior once again, now being able to weld workpieces with50% less material thicknesses in the bottom sheet.

The previous welds had fixed velocities during processing. The weldingagent of the present invention can take decisions with a clock rate of1,000 Hz, and it should therefore be able to adapt in real-time tovelocity variations while the processing takes place, as is demonstratedin FIG. 21. The moving device in this case is a robotic arm thataccelerates and slows down while the agent controls the ongoing weld.Because the imprecision of the robot creates a nonlinear velocityalteration when performing three different velocity levels within a 30cm workpiece, the final applied velocity has not been recorded. However,within the first 1.5 s of the process, the robot arm attempted to reachan approximate speed of 4.2 m/min, from 1.5 s to 4 s the robot attemptedto reach an approximate speed of 1.8 m/min, and for the remaining timeit attempted to reach an approximate speed of 6.6 m/min. For the firstspeed step, the welding agent applied an average 613 W, in the second412 W, and in the third 815 W. The workpiece scans show that no rootconvexity has been created, while the weld seam surface seemed to bemaintained over time. Therefore, the control agent shows comprehensiblebehavior.

As was mentioned before, although many process parameters influencelaser welding and cutting, as a rule of thumb the ratio of laser powerand processing velocity should develop in respect to an equal energy perunit length level, which should result in similar welding or cuttingresult. The simple models given in equations 2.1 and 2.2 state that thisratio should be approximately linear. In FIG. 22 the smoothed laserpower values are drawn vs. the processing speed. With this graph itbecomes obvious that the relation is almost linear within theexperiments carried out. In addition, the cross-sections shown in FIGS.16 and 17 show that the penetration depth is maintained approximately at1.1 mm±15%, while the velocity is altered by 650%. Therefore, it appearsas if closed-loop control laser power works successfully for theembodiments adaptively carried out in welding and cutting.

Thus, an advantage of the present invention concerning the processcontrol will be apparent: closed-loop control of laser power maintaineda penetration depth of approximately 1.1 mm±15%, while the welding speedaltered by 650%. This may significantly increase production output anddecrease system downtime.

Within the welding experiments that were carried out, the applied agentof the present invention managed to cope with six times the initialvelocity while maintaining a stable process. The agent fulfilled therequirement to preserve high connection without root convexity for thetested workpieces, except for varying thicknesses of material. When thebottom sheet was 50% thinner, the agent learned from additional feedbackprovided by a human expert to weld the new process kind within thestated requirements. Therefore, the welding agent displayed similarbehavior compared to the cutting agent discussed before. Both agentswere able to learn how to weld or cut from a human expert. They arerobust for the processes that they have been trained for, which may beadvantageous for industrial manufacturing. However, even when the agentshad to face new situations, they continued to show intelligible actions.When they have failed, they then responded positively to further humanexpert feedback. From the cognitive point of view, it is remarkable thatthe same agent design was able to learn how to cope with completelydifferent tasks, such as laser cutting on the one hand and laser weldingon the other. The next logical step toward greater cognitive capabilityis to reduce the necessity for human expert feedback and to create anagent that can act unsupervised.

In the following, a fourth embodiment of the present inventionconcerning unsupervised learning of welding processes will be described.

The previous description outlined how the cognitive agents learned fromhuman expert feedback. Thus the learning in the previous description wassupervised: human experts teach the agent positions within a trainingworkpiece where it should have increase or decrease laser power. Ifsomething undesirable happens, then the expert may provide feedback onhow to adjust laser power. However, it may be the case that no humanexpert is available to maintain the laser processing machine. It shouldbe possible for the cognitive system to learn from its own actions, orto give itself feedback. This kind of cognitive capability may beattained with reinforcement learning (RL). A classifier may take overthe role of giving feedback and provide a RL agent with rewards for itsown actions. The agent then learns a policy on how to act or how to weldbased on the feedback or on rewards received for its previousperformance. In order to test this, the learning task is therefore forthe agent to learn how to weld workpieces on the basis of gainedknowledge at different velocities without further human expertsupervision. Again, the desired weld seam should be well-fused, but itshould not be a full penetration weld. In this case it may still bepainted without a noticeable seam root trace afterwards.

In order to achieve the given learning task using reinforcementlearning, a reliable reward function is needed. As the system hasmultiple sensor data inputs, a classifier identifying features of a goodweld, such as a Support Vector Machine, may serve as reward functionr_(t), as is shown in FIG. 23. These rewards may fulfill the role of acritic in the Natural Actor-Critic method, which is described before. Asalso discussed before, the alteration of laser power equips the agentwith a tool or action both to influence the process outcomesignificantly and to allow for fast reactions during processing.Therefore, the next action that the agent chooses is absolute laserpower, a_(t). The chosen action depends on the learned policy, as isshown inπ(a _(t) |s _(t))=p(a _(t) |s _(t) ,w _(t)).  (Formula 4.1)

The policy parameters w_(t) relies on the gradient ĝ and w_(t−1), as inequation 2.25. However, for a full review of the applied algorithmplease consult the Natural Actor-Critic Algorithm with least-squarestemporal difference learning, LSTD-Q(λ). The policy should enable theagent to map from states, s_(t), to actions, a_(i), by learning fromrewards, r_(t). The rewards naturally influence the policy parameters.The best policy of the laser welding RL agent of the present inventionunder investigation has been found with a sigma function,

$\begin{matrix}{{{\pi\left( {\phi\left( {a_{t}❘s_{t}} \right)} \right)} = \left. {{L_{m}\frac{1}{1 + {\mathbb{e}}^{{- w_{t}^{T}}{\phi{(s_{t})}}}}} + \eta}\Rightarrow a_{t + 1} \right.},} & \left( {{Formula}\mspace{14mu} 4.2} \right)\end{matrix}$where L_(m) is the maximum allowed laser power and η is the explorationnoise determined by the product of a random number from −1 to 1 and theexploration parameter ε.

Put in simple words, the RL agent receives feature values and selectsthe next action in terms of laser power adjustment. The adjustment iscalculated using the policy parameters learned from a Support VectorMachine reward function, r_(t), giving the probability of good results.The policy parameters basically serve as weights for the feature valuesin order to compute the next action. Other parameters have to bepre-configured in order to apply the Natural Actor-Critic LSTD-Q(λ)algorithm. In the remainder of this section, the RL parameter is chosenas λ=0.4, which is required to calculate the statisticsz_(t+1)=λz_(t)+{circumflex over (φ)}(s_(t)). In order to computeequation 2.25, we need to choose the gradient ascent with a learningrate a in such a way that the gradient alteration shows an impact, butthe algorithm still has robust convergence, here for α=0.4. One of thethree policy parameters corresponds to the temperature diode feature,and the other two to camera feature values. The policy parameters behavesimilar to weights for the incoming feature set. For the initial stateof the policy parameters, we defined w₀=(5, 0, 0). Another startingvectors would have resulted in a different initial laser power, becausethe laser power is a linear combination of the incoming feature valuesand the policy parameters. In equation 4.2, the maximum laser powerL_(m) is set at 1,000 W. Furthermore, in order to calculate η, theexploration parameter ε is set to 0.5 W. Finally, the discount factor inequation 2.21 for future rewards has been chosen as γ=0.9.

In order to provide a reward function to the RL agent, a human expertgives initial feedback to the classifier from a training workpiece, asis demonstrated in FIG. 24. This training is different when compared tothe training workpieces mentioned above. The human expert feedback inthe previous description also contained information about laser powerthat was too high or too low for classes I, II, and III; within thistraining, the human expert simply provides the most basic informationabout good or bad feature values, which is stored as system knowledge inclasses A and B. The “good” class A is the area with maximal connection,but with no root convexity at the bottom sheet. Having a rewardfunction, the RL agent may learn a policy how it should map from statesto actions.

In the following, the RL agent learns from a set of experiments how tolap weld at three different velocities, 0.6 m/min, 1.2 m/min, and 1.8m/min. It was intended that all of the other process parameters werekept the same. Therefore, the RL agent should learn how to maintain asimilar energy per unit length level to achieve stable welds, butwithout root convexity at the bottom sheet. Every trial was initiatedwith the same RL parameter set that was described above.

In the first experiment with workpiece RIL004a, the weld takes place ata velocity of 1.2 m/min, as is shown in FIG. 25. The RL agent of thepresent invention starts with approximately 400 W laser power anddecides to raise this. Initially, the rewards are raising and the laserpower engages at a good level. Towards the end of trial RIL004a,although the laser power is the same due to common processirregularities, the power is too high and the laser beam is about tofully penetrate the workpiece, as can be seen in the light area of thescanned bottom sheet in FIG. 25. The RL agent continues the weld on thesame workpiece at a different starting point as a second trial RIL004b,shown in FIG. 26. This time, the policy parameters are initiated as theywere learned from the previous trial. It is noteworthy that the RL agentreceives sometimes good and sometimes medium rewards. The agent decidesto optimize its parameters slowly as it decreases the policy parameternamed “camera feature weight 1” compared to the other weights. However,the agent keeps the laser power at more or less same level because itcontinues to receive positive feedback. From the laser materialprocessing point of view, this weld was good and reliable although thelaser power was above optimum at the end of the processing time. The RLagent of the present invention reached a good power level and achievedthe goal of nearly maximum connection without excessive penetration orin other words an exiting beam at the bottom of the workpiece. However,looking at the rewards from a reinforcement learning point of view, thisoutcome may be acceptable but can be improved. In the following trialscontinuing the policy parameter development, the RL agent wouldpresumably have found a more suitable laser power; however, in thisconfiguration it may have taken several workpieces before the agent leftan area once it had received positive feedback.

The second workpiece RIL005 is welded at a velocity of 1.8 m/min, as isdemonstrated in FIG. 27. The RL agent again starts with 400 W laserpower, and finds itself in an area with appropriate laser power that isyet slightly below the optimum. The agent decides to raise the laserpower and then receives an optimal reward. The agent remains at thispower level and achieves an optimal welding result. The scans in FIG. 27show a brown line at the bottom sheet, (which could not be seen from thegrey-shaded FIG. 27, but has been observed in the experiments) whichindicates sufficient beam penetration but without root convexity. Thelaser power is just slightly higher than in the previous process withworkpiece RIL004. Obviously, the RL agent has chosen a laser power forthe RIL004 workpiece that is at the upper limit of the recommended laserpower area. However, the RL agent successfully learned the optimalpolicy parameters to weld this type of workpiece at this speed.

Finally, the RL agent has to learn how to weld a workpiece at asignificantly slower velocity of 0.8 m/min. Since the energy per unitlength should be approximately equivalent for similar welding results,we would hope that the RL agent chooses a lower laser power this time.As is shown in FIG. 28, starting with the defined initial policyparameter set, the agent begins with the laser power too high and thenmoves to a laser power that is too low. The reward varies over this timeand the RL agent alters the policy parameters until it finds aconfiguration with raising rewards. Later, the rewards tend towards anoptimal result and the RL agent converges at this energy level. Afterconvergence, the weld seam is as desired and the RL agent has learned agood laser power for this welding velocity by processing just oneworkpiece.

Thus, an advantage of the present invention with respect to cognitivecapabilities will be apparent: a RL agent learns how to weld atdifferent speeds in situations it has not been trained for.

The parameters for optimal laser welding results in industrialmanufacturing are mainly established through several manual trials. Ifthe process or task varies from workload to workload, costly manualtrials have to be repeated. The RL agent shows the capacity to learnsuitable welding process parameters, potentially during the first trialof each new process. It is remarkable that the RL agent has no priorknowledge of which action or laser power would be appropriate, and yetfinds comprehensible solutions. Within workpiece RIL004, the RL agenthas managed to find an adequate laser power despite a noisy rewardsignal. Because the RL agent has learned in the demonstrated trials howto weld this kind of workpiece at different velocities, or how to choosea specific action such as “apply 360 W!” out of a large group ofpossible choices, the investigated machine seems to have something likea cognitive capability.

In summary, it has been described how cognitive capabilities can berealized within a processing procedure approach for production machines.A procedure for creating agents using different modules of a cognitivearchitecture has been outlined. The cognitive capabilities of the agentsaccording to the present invention are as follows: the agents canabstract process relevant information using dimensionality reduction;they can extract features autonomously; and they can learn from humanexpert feedback how to connect the feature values in order to fulfillmonitoring or processing tasks. Using dimensionality reduction ratherthan raw sensor data decreases the sensor data volume significantly,from approximately 30 MBytes to just 10 kBytes per second, and enablesthe agent to act quickly. This is possible due to different agentdesigns incorporating techniques such as Isomap and Fuzzy KNN.Furthermore, the system's adaptability is increased through detectingsimilarities within the feature space. This leads to a self-learningconcept, meaning that the agent can successfully map features from aknown training workpiece onto a previously unknown workpiece. In thisway, the agent is able to learn unsupervised.

The first object concerning cognitive capabilities has been achieved.The system abstracts relevant information by reducing the incoming rawsensor data to a thousandth, still capable of monitoring. Differentagent designs enable two learning modes: supervised and unsupervised.Demonstrations show that the agents can learn from a human expert andtransfer knowledge, for instance, how to cut a new workpiece almost justhalf as thick. A RL agent learns how to weld at different speeds insituations it has not been trained for.

The experimental results validated the cognitive capabilities of thesystem, showing that it can accomplish previously challenging monitoringand process control tasks. In a laser welding experiment, a monitoringagent successfully detects a lack of fusion within lap joints ofzinc-coated steel sheets.

The second object concerning monitoring has been achieved by asuccessful detection of false friends that occurred because of insertedgaps of 1.0 and 0.6 mm within zinc-coated steel lap welds. A reliabledetection may increase quality in car body production.

In laser cutting experiments, a control agent uses the improvedmonitoring signal to close the loop to the laser power control. Itchanges the laser power using cutting speed alterations and maintains aminimum kerf width for materials such as mild steel and stainless steel,even though it was trained for stainless steel only. Furthermore,another agent shows its adaptive skills and learns how to weld withoutexcessive penetration from a human expert.

The fourth object concerning adaptability has been achieved. The agentsadapt to various situations, two different production processes such ascutting and welding at different speeds; 50% less material thickness inthe bottom sheet in welding after additional training; three differentmaterials with laser cutting such as zinc-coated steel, mild steel, andstainless steel.

Cross-sections of the agent controlled welds show that it maintained apenetration depth of approximately 1.1 mm±15%, despite the fact that thewelding speed altered by 650%. Further analysis showed that theexperimental closed-loop control results comply with theoretical modelsfor laser cutting and welding from the literature.

The third object concerning process control has been achieved.Closed-loop control of laser power maintained a penetration depth ofapproximately 1.1 mm±15%, while the welding speed altered by 650%. Thismay significantly increase production output and decrease systemdowntime.

A reinforcement learning agent also showed in experiments that it couldlearn unsupervised from its own feedback about how to weld at differentvelocities.

Summary and Conclusions

In order to increase the flexibility, quality, and output of productionmachines, the present invention has investigated an architecture thatcreates software agents with cognitive capabilities. These agents canlearn from human experts how to weld to a penetration depth of 1.1 mm,or how to maintain a minimum kerf width during cutting. Usingdimensionality reduction, classification, and reinforcement learning,they are able to provide themselves with feedback. Within an industrialenvironment they can monitor lack of fusion in zinc-coated steel lapweld experiments. Furthermore, in the closed-loop real-time control oflaser power, they maintained the processing goal to within ±15% whilethe speed altered at 650%.

The present invention has investigated modules that are suitable for acognitive architecture for production machines within a cognitiveperception-action loop connecting sensors and actuators. As anindustrial scenario, it develops a procedure for laser materialprocessing that can create artificial agents for laser welding orcutting with cognitive capabilities. These capabilities are: to abstractrelevant information; to learn from a human expert; to use the gainedknowledge to make decisions; and to learn how to handle situations thatthe agent has not previously been trained in. The cognitive agentsachieve improved processing results within the chosen scenarios withthese capabilities.

It is possible to use laser beams when processing materials forefficient joining and cutting in a wide range of applications. From amanufacturing point of view, laser material processing is often the bestchoice for high production volumes. In terms of mass customization,current trends demand greater flexibility for the production techniquesof the future. This is a special challenge for laser material processingbecause great efforts and high costs are necessary before quality weldsand cuts can take place. A system that is capable of learning how toweld or cut has the potential to increase flexibility and thus the rangeof laser applications. Furthermore, the superior data analysiscapabilities of artificial agents may enable them to comprehend incomingsensor data and establish improved monitoring or even process controlabilities.

Many sophisticated approaches to monitoring and process control systemshave been described in the literature. However, many of these could notbe transferred to real manufacturing situations for different reasons;for example, they provide solutions to individual processes that cannotbe converted to a high number of different applications, or they sufferfrom noisy sensor data.

The cognitive architecture according to the present invention integratesseveral sensors, such as photodiodes, acoustic sensors, and cameras. Thedata is reduced in dimension, enabling the agent to handle large amountsof raw sensor data represented in significantly fewer features throughalmost the same information content, in this case from approximately 30MBytes to just 10 kBytes per second. Furthermore, the agent can identifysimilarities within the features and cluster them within a givenclassification, which satisfies the first object. This creates animproved monitoring signal that is suitable for detecting a lack offusion or false friends within the zinc-coated steel welds, as testedand fulfilling the second object. For the tested data, the Isomapalgorithm outperformed LDA and PCA, enabling us to use comparativelysimple classifiers such as Fuzzy KNN. A control agent can close the loopand connect the monitoring signal using laser power control. Thisenables the investigated system to maintain the desired welding orcutting results within a broad range of different processing speeds,material types, and thicknesses. As the welding speed alters by about650%, the agent maintains a penetration depth of 1.1 mm±15%, whichreaches the third object. A comparison of the cutting and weldingresults with simple analytical models also shows that the systemexhibits reasonable behavior in closed-loop control because the ratio oflaser power and process velocity is almost linear. The design of thecognitive architecture enables agents to process the high volume ofsensor data quickly enough to handle fast processing speed alterationsor jumps in material thickness toward the fourth object. Moreover, usingreinforcement learning, the agent managed to learn new parameter setsfor situations that it had not been trained for.

It appears that the present invention is one of the first to discusssome of the techniques mentioned above in relation to laser welding orcutting. Furthermore, the present invention delivers real-worldcomparisons of these techniques, which have often tested only withartificial data sets. Within the experiments, the defined cognitivecapabilities have been shown to enable production systems to improvetheir processing tasks in flexibility, quality, and efficiency. It isnoteworthy that machines use an ability such as learning to entitle themto do this.

The present invention has demonstrated steps towards automatingproduction machines, using laser material processing with cognitiveagents. The embodiments showed improved processing in some monitoringand closed-loop control tasks.

As described in the above embodiments, the agents mainly controlledlaser power; however, it is also preferred by the present invention tocontrol gas pressure, focal position, processing speed, or a combinationof these. Furthermore, the agents have only been described for onedirectional welding or cutting. The concept may work well for movementsof processing optics relative to a workpiece in a two orthree-dimensional space. Optical flow detection, when applied to theilluminated in-process pictures, should be able to deliver atwo-dimensional vector describing this movement.

An agent may be trained to detect many different processing defects,such as a mismatch between the keyhole and the desired weld position.Another promising approach of the present invention would be to combineremote laser welding or cutting with the cognitive architectureproposed.

From a data processing point of view, the architecture of the presentinvention allows switching between dimensionality reduction techniques,classification methods, and self-learning mechanisms, as well asevaluating the combined performance. Other data reduction or sensingmethods could improve feature extraction, and the next steps would becompressive sensing or random projections. Further work towardsefficient out-of-sample extension and increased Isomap featuretransparency would strengthen the underlying concept. Since thiscognitive architecture has demonstrated reliable learning anddecision-making capabilities for individual processing, it may bedesirable to extend the architecture to multi-agent learning concepts.Currently, the sensor units used in laser welding may be categorizedinto pre-processing, in-processing, and post-processing, all monitoringthe same process. Further process improvements may be accomplished byintegrating team learning and processing goal achieving strategies usingmultiple agents. A multiple agent strategy could also allow informationsharing and learning within different workstations distributed atdifferent locations. If a system learns how to handle a task, a muchlarger group can gain simultaneously from it.

As described above, it is possible to detect the movement of aprocessing head or optical system relative to the workpiece with opticalflow detection. An example for illustrating the principle of thistechnique of optical flow detection is described in Horn and Schunk“Determining optical flow”.

The video data of a camera or suitable optical sensor data may beanalyzed with the method of optical flow detection resulting in one orseveral vectors pointing toward the direction of the relative movement.By comparison of successive image frames, a translation and/or rotationwithin the 2D-image plane could be detected. Thus, taking additionallytime delay between the successive frames into account, a velocity/matrixvector could also be calculated. In summary, the relative attitude ofthe laser processing head relative to the workpiece at each time couldbe detected by means of optical flow detection.

The method of optical flow detection could be employed by the processinghead of the present invention, since the workpiece is illuminated byLEDs or laser sources having a different wavelength than the processinglaser. Further, optical filter systems could be employed to preventdisturbances generated by the light of the processing laser beam orgenerated by the emission of the molten material of the workpiece(process light).

The method of optical flow detection could be applied to a digital imageprocessing of the complete image frame or of a part or section of thesuccessive image frames. In addition, a separate sensor could beemployed, which has an illumination source at its own or uses the lightilluminating the workpiece generated by the laser processing head.

The method of optical flow detection could be applied to two differentfunctionalities.

The first feature is to compensate a mismatch of a keyhole and a desiredweld position, when the geometry of the keyhole is detected for afurther classification by the cognitive laser processing head. By usingthe optical flow detection method, a rotation or translation of theprocessing head could be detected, in that the movement vector iscalculated by the optical flow detection method, and this movementvector may be used to determine a degree a with regard to the featuredirection vector, which is the normalized vector of the featurecalculation method. Any face recognition methods could improve thefeature extraction.

The second feature of the optical flow detection method employed in alaser processing head of the present invention is to compensate amovement of a laser processing head actuated by an actuator to generatea welding or cutting line having an exact predetermined geometry as setin advance of the laser machining process. Such compensation could bedone by an actuator of the focusing lens or a mirror within the lasermachining head, for example already known by thewobble-tracker-technique. Thus, a displacement or shift of thepredetermined track of the laser machining head is recognized by theoptical flow detection and compensated by an actuator actuating themirror reflecting the laser in a direction laterally to a movementdirection, or in particular in a perpendicular direction, to generate aperfect welding line or cutting line.

In the following, a preferred fast and very effective method fornormalizing the orientation of a multitude of images recorded by acamera observing the processing area of a laser processing operationwill be described.

In a first step, an extraction of a threshold image is performed. Thethreshold for each pixel of the video is extracted by capturing a videoof the workpiece's surface illuminated by a light source as describedabove with regard to the optical flow detection. The video can either beacquired before the laser is turned on, or alternatively over the wholeworkpiece without the processing laser. The mean image of this video isused as the threshold on a per pixel basis. Alternatively a fixedthreshold for all pixels can be used. The images of the process arecaptured using a high speed camera that is mounted coaxially to theprocessing head. Multiple exposure times can be used to improve theperformance of the detection of the seam/kerf and the laser spot.

In a second step, the laser position within the captured images isextracted. The laser spot can be observed as the brightest values on theimage, especially on low exposures times. In the trials could resultswere achieved with using the upper 30% of the maximum value range of theimage sensor as a threshold for binarization. The binary image is thenfurther reduced by calculating the centroid of the image P_(LP)(x_(LP),y_(LP)).

Further, extraction of the weld seam in case of a laser welding processor a cutting kerf in case of a laser cutting process is performed. Theacquired video is binarized using the threshold image created in thefirst step. Pixels above 60% of the threshold are white, pixels beloware black. The weld seam or the cutting kerf can be observed as a blackline in the images. Noise and holes in the images are removed by binaryerosion, followed by a binary dilation. The structuring elements ofthese operation can be chosen independent of each other in order toimprove the denoising performance. The images are than inverted, meaningthe workpiece's surface will become dark and the seam/kerf white. Thiswhite line in the image is then reduced by calculating the centroid ofthe image. Alternatively, only the centroid of this Blob can be computedin order to improve the detection performance. This results in a singlepoint P_(S)(x_(S),y_(S)).

After the preceding steps, laser spot position normalization and theextraction of the rotation angle could be performed, which gives acomplete information of attitude of each image with respect to theposition of the laser spot in comparison to the generated weldseam/cutting kerf. The position of the laser spot in the image isnormalized by a shift of the image by a vector P_(C)−P_(LP). P_(C) isthe center of the picture and P_(LP) is the position of the laser spot.Therefore the laser spot is centered in the middle of the picture afterthis step. The rotation angle α can be extracted by transforming thecoordinates of P_(S) into a polar coordinate system with its center atP_(C). The angle α is calculated by a tan 2(y_(S)−y_(C), x_(S)−x_(C)).After the rotation of −α around the image centered the weld seam/cuttingkerf is pointing to the right image border, independent of the processdirection and the position of the laser spot.

The thus normalized images, taken for example in a training trial forgenerating a training data set or during a processing operation to beclassified or closed-loop controlled, could then be used for calculatinga feature vector with reduced dimensionality by means of a PCA or LDAoperation to be used for a classification of the feature vector.

In the following, procedures for interacting entities of manufacturingprocesses according to the present invention should be described. Thehighest share of world wide active robots and automated systems serve inmanufacturing. However, state of the art industrial systems may still beimproved in skills such as fast learning or reliable adaptation to newsituations. We have thoroughly investigated and developed “perception”,“cognition”, and “action” (P-C-A) loops, cognitive agents, and machinelearning techniques suitable for industrial processes with actuators andintelligent sensors. Transferring cognitive capabilities, knowledge, andskills, as well as creating many interacting P-C-A loops is our next aimtowards the cognitive factory.

Only very few industrial production processes are unique. The majorityof production processes run at different facilities or at differenttimes performing identical tasks in similar environments. Still, oftenno or limited information exchange exists between these processes. Thesame workstations often require an individual configuration of everyentity managing similar process tasks. Typical examples of suchensembles are spatiotemporally separated machines of car body productionlines or treatments in the chemical industry. In order to increase thecapability of machines to help each other we will combine in space ortime distributed P-C-A loops. Certain topics arise to approach this aim:In order to enable skill transfer between different entities we need toestablish a reliable and adaptable Multi-P-C-A-loop topology. Thismeta-system should be able to identify similar processes, translatesensor data, aquire features, and analyze results of the differententities. Dimensionality reduction, clustering, and classificationtechniques may enable the machines to communicate on higher levels.Machine-machine trust models, collective learning, and knowledgerepresentation are essential for this purpose. Furthermore someindustrial processes may be redefined to optimize the overallperformance in cognitive terms. Both data processing and hardwareconfiguration should result in a secure, reliable, and powerfulprocedure to share information and transfer skills between differentunits of one production cell or assembly line of distributed factorylocations.

Using self-optimizing algorithms for control or parameterization ofindustrial applications offers the possibility to continuously improvethe individual knowledge base. Reinforcement learning, for instance,gives a set of methods that provide this possibility. These algorithmsrely on exploration in the processes state-space in order to learn theoptimal state-action combinations. A reinforcement learning agent canalso be described by a simple P-C-A-Loop, where the process ofevaluating the state information of the environment is the “perception”element of the loop, the alteration of current control laws representsthe “action” part and the process of mapping estimated state informationto new control laws gives the “cognition” section of the single P-C-Aloop. In industrial applications exploring a large state-space is notalways feasible for various reasons like safety, speed, or costs. Usingthe Multi-P-C-A-Loop approach for distributing the learning task overmultiple agents, can reduce the amount of exploration for the individualagent, while the amount of learning experience still remains high. Itfurthermore enables the teaching among different P-C-A loops. A possibleassignment for the Multi-P-C-A approach is the combination of multipleagents in one system or assembly line, for instance a monitoring and aclosed-loop control unit. Two different agents could be trained foroptimization of different process parameters. The combination of both ona Multi-P-C-A level could be used to find an optimal path for allparameters.

Both outlined Multi-P-C-A-Loops may improve manufacturing performance insetup and configuration times, process flexibility as well as quality.One approach combines and jointly improves similar workstations withjoint knowledge and skill transfer. The other enables different units toself-improve with each others feedback.

In the following, a networking system for cognitive laser processingdevices according to the present invention shouls be described. There isa growing demand for autonomous industrial production systems withincreased flexibility, especially in countries with high labor costs.Because laser welding processes are individually different in opticalsetup, materials, or joint geometry, the current laser welding systemshave to be configured with many manual trials by human experts. Onceconfigured industrial laser welding systems require costly manualreconfiguration for every process change. To expedite the setup andreconfiguration times human experts often use tables and knowledge fromprevious work to take a good guess of initial process parameters. Evenwhen fully configured small undesired laser welding process variationsmay have a large impact on the seam quality.

In order to improve laser welding quality, increase automation andflexibility as well as reduce costs of configuration and down times wewant to apply modern machine learning methods. Our recent researchresults indicate that cognitive laser welding systems equipped withmachine learning can learn laser welding parameters from human expertfeedback. The systems improve with every feedback iteration but needenough training data to improve processing.

It is an object of the present invention to provide a Networking systemfor cognitive laser processing devices and a cognitive laser processingdevice being adapted to communicate with the Networking system, by whichthe productivity of each laser processing device in the Networkingsystem is enhanced.

This object is solved by a Networking system for cognitive laserprocessing devices and by a cognitive laser processing device beingadapted to communicate with the Networking system.

In particular, the present invention is directed to a Networking system,in which training data of a plurality of cognitive laser processingdevices connected to the Network System is jointly collected andanalyzed on a large scale how to laser process, in particular laserweld, individually different workpieces under different processenvironments.

It is an advantage of the present invention, that, once thecollaborative systems gain enough machine knowledge, they avoidrepetitive configuration steps and may significantly reduce down timesas well as increase product flexibility.

According to one embodiment of the present invention, in order tofacilitate the integration of several cognitive control systems, alldistributed systems are connected to each other via internet. Theknowledge gained by these systems is shared, thus allowing a globaldatabase of process configurations, sensor setups and qualitybenchmarks.

In order to share information between machines, all of them have to usea similar method of feature acquisition. Different laser weldingscenarios are constantly being investigated within labs located ondifferent places on the world. Within this consortia of labs, we canacquire the necessary training data and processing knowledge for alocally distributed network of cognitive laser welding systems of thefuture. Further participants within this network to come can contributeand benefit from the automatically growing machine knowledge.

As a first scenario to achieve these goals using cognitive dataprocessing approaches for combining the input data from multiple sensorsin order to receive a good estimation of the state the process iscurrently in. The systems will be composed of a coaxially mountedcamera, photodiodes, and an optical interferometric sensor. The camerawill provide information about the melt pool and keyhole geometries,while the photodiodes are giving a very high spectral resolution ofoptical emissions. The interferometric sensor can provide pre- andpost-process data.

Using cognitive dimensionality reduction techniques, unnecessary andredundant data from these sensors can be removed. The reduced sensordata is used to classify the state of the process. Clustering allows foridentification of specific process states, even between differentset-ups. If a significant difference from the references, and thereforean unknown process condition, is detected, the supervisor will bealerted. The expert can then teach the new state and countermeasures (ifpossible) to the system in order to improve its performance.

The cognitive system to be developed should be able to learn to separateacceptable and unacceptable results and furthermore be able to avoidunacceptable results where possible. The usage of technical cognitioneliminates the need for a complete physical model of the welding orcutting process. The system is able to stabilize the process byimproving at least one steering variable. Distributed cognition allowsfor a central database between different manufacturing locations. Theinformation gathered from one process can be transferred to a similarprocess at a different location.

The learning abilities of the system together with the ability to shareand cluster the knowledge between manufacturing locations significantlyreduces the expert time needed for calibration, leading to an improvedthroughput, higher agility and lower production costs.

According to the present invention, the efficiency in environments,where laser material processing is already successfully used, isimproved, while increasing the potential market of laser applications toareas where it has not been used due to quality and reliabilityconcerns. The cognitive laser welding network will offer two significantadvantages to industrial laser welding: it can autonomously process abroad set of different laser welding scenarios and the joint knowledgewill exponentially improve over time for all future participants in thisnetwork. It should be emphasized that the above described system andprocesses could also employed to a wide field of laser machiningprocesses like laser cutting, laser ablation, laser converting, laserdrilling, laser engraving, or laser soldering.

According to the present invention, a method is provided, which is usedfor monitoring a laser welding process for detecting a lack of fusion ofworkpieces to be joined, comprising the steps of: (a) recording a pixelimage at an initial time point displaying the interaction zone between alaser beam and the workpieces by means of a camera; (b) converting thepixel image into a pixel vector; (c) representing the pixel vector by asum of predetermined pixel mappings each multiplied by a correspondingfeature value; (d) classifying the set of feature values on the basis oflearned feature values for determining a lack of fusion between theworkpieces to be joined at the initial time point; and (e) repeating thesteps (a) to (d) for further time points to perform a monitored laserwelding process. In this method the predetermined pixel mappings arepreferably obtained by the steps of: recording a training set of pixelimages at a predetermined first number of time points displaying theinteraction zone having no lack of fusion between training workpieces tobe joined, and at a predetermined second number of time pointsdisplaying the interaction zone having a lack of fusion between thetraining workpieces to be joined; converting the pixel images into pixelvectors and generating a covariance matrix from the pixel vectors;calculating the eigenvectors of the covariance matrix to determine thepredetermined pixel mappings. In this method the learned feature valuesare preferably obtained by the following steps: representing each pixelvector of the training set by a sum of predetermined pixel mappingsmultiplied by corresponding feature values; and training a classifierwith the obtained feature values by discriminating feature values at thefirst number and the second number of time points. In this method theclassifier is selected from a group comprising Support Vector Machines(SVM), Artificial Neural Networks (ANN), or a Fuzzy-KNN. In this methoda further feature value is preferably obtained by measuring thetemperature of the interaction zone by means of an optical temperaturesensor. In this method the recorded pixel image of the camera is agrey-scale pixel image. In this method the recorded pixel image isalternatively a color image of the camera, wherein each color sub-pixelimage is converted to a separate pixel vector used for classification.In this method the predetermined pixel mappings are calculated by meansof isometric feature mapping (ISOMAP), linear discriminant analysis(LDA) and principal component analysis (PCA). This method preferablyfurther comprises the step of outputting an alert signal, if a lack offusion between the workpieces to be joined is determined. This methodpreferably further comprises the step of controlling an actuator on thebasis of the classification result. In this method preferably theactuator is a laser power control or a processing speed control. In thismethod the workpieces to be joined are preferably two zinc-coatedworkpieces having a gap in between. There is preferably provided a Lasermaterial processing head having a control unit being adapted to performthe above method. The Laser material processing head preferablycomprises a high-speed camera, sensors for solid-borne and air-borneacoustics, a temperature sensor and three photodiodes recording processemissions on different wavelengths for generating sensor data to be usedas feature values. The Laser material processing head preferably furthercomprises a PID-unit for controlling laser power on the basis of theclassification result.

According to the present invention, a Method for controlling aprocessing operation of a workpiece by means of a Reinforcement Learning(RL) agent unit is provided, comprising the steps of: (a) observing aninteraction zone in the workpiece by means of at least one radiationsensor to generate at least one sensor signal s_(t), wherein theworkpiece is processed using an actuator having an initial actuatorvalue a_(t); (b) determining a basis function φ(s_(t)) from the set ofsensor signals s_(t); (c) determining a reward function r_(t) giving theprobability of good results of the processing operation; (d) choosing anext actuator value a_(t+1) on the basis of a policy π depending on thereward function r_(t) and the basis function φ(s_(t)); and (e) repeatingthe steps (a) to (d) for further time points to perform a RL controlledprocessing operation. 2. Method according to embodiment 1, wherein steps(a) and (b) comprise the steps of:—recording a pixel image at an initialtime point of an interaction zone in the workpiece by means of a camera,wherein the workpiece is processed using an actuator having an initialactuator value a_(t); and—converting the pixel image into a pixel vectors_(t): and—representing the pixel vector s_(t) by a sum of predeterminedpixel mappings each multiplied by a corresponding feature value, whereinthe set of feature values φ(s_(t)) represents a basis function φ(s_(t)).3. Method according to embodiment 1 or 2, wherein step (c) comprises thestep of classifying the set of feature values φ(s_(t)) on the basis oflearned feature values to determine the reward function r_(t). 4. Methodaccording to embodiment 1, 2 or 3, wherein the step (c) comprises thestep of classifying a set of sensor data measured by a post-processsensor and/or a pre-process sensor. 5. Method according to one of thepreceding embodiments, wherein the actuator value a_(t) is the laserpower of a processing laser beam interacting with the workpiece in theinteraction zone or the processing velocity. 6. Method according to oneof the preceding embodiments, wherein the processing operation is alaser processing operation comprising a laser welding process or a lasercutting process. 7. Method according to embodiment 2, wherein thepredetermined pixel mappings are obtained by the steps of:—recording atraining set of pixel images at a predetermined first number of timepoints displaying the interaction zone having a good processing result,and at a predetermined second number of time points displaying theinteraction zone having a bad processing result;—converting the pixelimages into pixel vectors and generating a covariance matrix from thepixel vectors;—calculating the eigenvectors of the covariance matrix todetermine the predetermined pixel mappings. 8. Method according toembodiment 7, wherein the learned feature values are obtained by thefollowing steps:—representing each pixel vector of the training set by asum of predetermined pixel mappings multiplied by corresponding featurevalues; and—training a classifier with the obtained feature values bydiscriminating feature values at the first number and the second numberof time points into a good class and a bad class. 9. Method according toembodiment 8, wherein the classifier is selected from a group comprisingSupport Vector Machines (SVM), Artificial Neural Networks (ANN), or aFuzzy-KNN. 10. Method according to one of the preceding claims, whereina further feature value φ(s_(t)) is obtained by measuring thetemperature of the interaction zone by means of an optical temperaturesensor. 11. Method according to one of the preceding embodiments,wherein further feature values φ(s_(t)) are obtained by sensor data ofthree photodiodes recording process emissions on different wavelengths.12. Method according to one of the preceding embodiments, wherein therecorded pixel image of the camera is a pixel image of the processedworkpiece illuminated by LEDs or laser sources having a differentwavelength than the processing laser, wherein further optical filtersystems are employed to prevent disturbances generated by the light ofthe processing laser beam or generated by the emission of the moltenmaterial of the workpiece. 13. Method according to embodiment 7, whereinthe predetermined pixel mappings are calculated by means of isometricfeature mapping (ISOMAP), linear discriminant analysis (LDA) orprincipal component analysis (PCA). 14. Laser material processing headhaving a Reinforcement Learning agent unit being adapted to perform amethod according to one of the preceding embodiments. 15. Laser materialprocessing head according to embodiment 14, comprising a high-speedcamera, sensors for solid-borne and air-borne acoustics, a temperaturesensor and three photodiodes recording process emissions on differentwavelengths for generating sensor data to be used as feature values.

According to the present invention, a method for closed-loop controllinga processing operation of a workpiece is provided, comprising the stepsof: (a) recording a pixel image at an initial time point of aninteraction zone by means of a camera, wherein the workpiece isprocessed using an actuator having an initial actuator value; (b)converting the pixel image into a pixel vector; (c) representing thepixel vector by a sum of predetermined pixel mappings each multiplied bya corresponding feature value; (d) classifying the set of feature valueson the basis of learned feature values into at least two classes of agroup of classes comprising a first class of a too high actuator value,a second class of a sufficient actuator value and a third class of a toolow actuator value at the initial time point; (e) performing a controlstep for adapting the actuator value by minimizing the error e, betweena quality indicator y_(e) and a desired value; and (f) repeating thesteps (a) to (e) for further time points to perform a closed-loopcontrolled processing operation. 2. Method according to embodiment 1,wherein the quality indicator y_(e) is represented by the difference ofthe class probability of a current set of feature values being part ofthe third class and the class probability of a current set of featurevalues being part of the first class. 3. Method according to embodiment1 or 2, further comprising the step of varying the desired value by auser during the processing operation to optimize a desired processresult. 4. Method according to embodiment 1, 2 or 3, wherein the controlstep comprises adapting the actuator value at a respective time point tby means of a PID control output c₁, which is represented by

$c_{t} = {{Pe}_{t} + {I{\sum\limits_{i = {t - n}}^{t - 1}e_{i}}} + {D\left( {e_{t} - e_{t - 1}} \right)}}$with P for proportional, I for integral, and D for derivative behaviour.5. Method according to one of the preceding embodiments, wherein theactuator value is the laser power of a processing laser beam interactingwith the workpiece in the interaction zone or the processing velocity.6. Method according to one of the preceding embodiments, wherein thelaser processing operation is a laser welding process, a laser cuttingprocess, a laser soldering process, a laser hybrid welding process, or alaser cladding process. 7. Method according to one of the precedingembodiments, wherein the predetermined pixel mappings are obtained bythe steps of:—recording a training set of pixel images at apredetermined first number of time points displaying the interactionzone having a too high actuator value, at a predetermined second numberof time points displaying the interaction zone having a sufficientactuator value, and at a predetermined third number of time pointsdisplaying the interaction zone having a too low actuatorvalue;—converting the pixel images into pixel vectors and generating acovariance matrix from the pixel vectors;—calculating the eigenvectorsof the covariance matrix to determine the predetermined pixel mappings.8. Method according to embodiments 7, wherein the learned feature valuesare obtained by the following steps:—representing each pixel vector ofthe training set by a sum of predetermined pixel mappings multiplied bycorresponding feature values; and—training a classifier with theobtained feature values by discriminating feature values at the firstnumber, the second number, and the third number of time points. 9.Method according to embodiment 8, wherein the classifier is selectedfrom a group comprising Support Vector Machines (SVM), Artificial NeuralNetworks (ANN), or a Fuzzy-KNN. 10. Method according to one of thepreceding embodiments, wherein a further feature value is obtained bymeasuring the temperature of the interaction zone by means of an opticaltemperature sensor and/or by sensor data of three photodiodes recordingprocess emissions on different wavelengths 11. Method according to oneof the preceding embodiments, wherein the recorded pixel image of thecamera is a pixel image of the processed workpiece illuminated by LEDsor laser sources having a different wavelength than the processinglaser, wherein further optical filter systems are employed to preventdisturbances generated by the light of a processing laser beam orgenerated by the emission of a molten material of the workpiece. 12.Method according to one of the preceding embodiments, wherein thepredetermined pixel mappings are calculated by means of isometricfeature mapping (ISOMAP), linear discriminant analysis (LDA) orprincipal component analysis (PCA). 13. Laser material processing headhaving a control unit being adapted to perform a method according to oneof the preceding embodiments, wherein the actuator value is the laserpower of a processing laser beam interacting with the workpiece in theinteraction zone or the processing velocity. 14. Laser materialprocessing head according to embodiment 13, comprising a high-speedcamera, sensors for solid-borne and air-borne acoustics, a temperaturesensor and three photodiodes recording process emissions on differentwavelengths for generating sensor data to be used as feature values. 15.Laser material processing head according to embodiment 13 or 14, furthercomprising a PID-unit for controlling laser power on the basis of theclassification result.

The invention claimed is:
 1. A method for controlling a processingoperation of a workpiece by means of a Reinforcement Learning (RL) agentunit, comprising the steps of: (a) observing an interaction zone in theworkpiece by means of at least one radiation sensor to generate at leastone sensor signal S_(t), wherein the workpiece is processed using anactuator having an initial actuator value a_(t), wherein the positionand orientation of the keyhole in the recorded pixel image is normalizedaccording to a method for classifying a multitude of images recorded bya camera observing a processing area of a workpiece processed by aprocessing beam, comprising the steps of recording a first pixel imageand a multitude of subsequent pixel images by the camera during aprocessing operation, detecting mismatches of a position and orientationof a keyhole generated by the processing beam in the workpiece within animage plane of the subsequent pixel images in comparison to the firstpixel image, compensating the mismatches of the position and orientationof the respective keyholes in the subsequent pixel images with regard tothe first pixel image, to produce a set of pixel images having each anormalized keyhole position and orientation, classifying the set ofnormalized pixel images into at least two classes by means of aclassifier; (b) determining a basis function φ(S_(t)) from the set ofsensor signals S_(t); (c) determining a reward function r_(t), givingthe probability of good results of the processing operation; (d)choosing a next actuator value a_(t+1) on the basis of a policy adepending on the reward function r_(t) and the basis function φ(S_(t));and (e) repeating the steps (a) to (d) for further time points toperform an RL controlled processing operation.
 2. A method forclosed-loop controlling a processing operation of a workpiece,comprising the steps of: (a) recording a pixel image at an initial timepoint of an interaction zone by means of a camera, wherein the workpieceis processed using an actuator having an initial actuator value, whereinthe position and orientation of the keyhole in the recorded pixel imageis normalized according to a method for classifying a multitude ofimages recorded by a camera observing a processing area of a workpieceprocessed by a processing beam, comprising the steps of recording afirst pixel image and a multitude of subsequent pixel images by thecamera during a processing operation, detecting mismatches of a positionand orientation of a keyhole generated by the processing beam in theworkpiece within an image plane of the subsequent pixel images incomparison to the first pixel image, compensating the mismatches of theposition and orientation of the respective keyholes in the subsequentpixel images with regard to the first pixel image, to produce a set ofpixel images having each a normalized keyhole position and orientation,and classifying the set of normalized pixel images into at least twoclasses by means of a classifier; (b) converting the pixel image into apixel vector; (c) representing the pixel vector by a sum ofpredetermined pixel mappings each multiplied by a corresponding featurevalue; (d) classifying the set of feature values on the basis of learnedfeature values into at least two classes of a group of classescomprising a first class of a too high actuator value, a second class ofa sufficient actuator value and a third class of a too low actuatorvalue at the initial time point; (e) performing a control step foradapting the actuator value by minimizing the error e_(t) between aquality indicator y_(e) and a desired value; and (f) repeating the steps(a) to (e) for further time points to perform a closed-loop controlledprocessing operation.